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FOREWORD 


The purnose of this handbook is to provide a text ar.d reference material 
in System Analysis and Cost-Effectiveness . It is intended for those technical, 
scientific, management, and administrative personnel who are responsible for 
preparing information, making decisions or reviewing decisions made by others 
regarding life-cycle cost, system effectiveness (availability, dependability, 
capability), or technical feasibility of a system or equipment at any phase 
in its life cycle. It is immedi at^ly useful to personnel who are familiar 
with ?. system or equipment under study but are not familiar with the method- 
ology and techniques of System Analysis and Cost-Effectiveness. 

The handbook consists of four chapters: (1) an introduction to the con- 
cept of system analysis and coct-ef fectiveness ; (2) a basic framework, or 
general methodological approach, for conducting and reviewing cost-effectiveness 
or system analysis studies; (3) a set of techniques (linear programming, queue- 
ing theory, simulation, etc.) that can bo used for performing cost-ef fectiveness 
and system analysis studies; and (4) a review of the basic mathematical and sta- 
tistical concepts that underlie the scientific approach in the system analysis/ 
cost-effectiveness process. 

The handbook was originally written by ARINC Research Corporation, 2551 Riva 
Road, Annapolis, Maryland 21401, in response to line item 0003 Exhibit 4002 of 
Contract Number DAAB07-6P-C-0056 for the Systems/Cost Analysis Office. U. P Army 
Electronics Command, Fort Monmouth, New Jersey 07703, Messrs. J. A. Maclnko 
and R. J. Sanford were the USAEC0M Project Engineers and Mr. D. p. Salvano, Chief, 
Systems Evaluation Division, was the Project Advisor. It Is now bring published 
as an AMC handbook in this series designated AMCP 706-191 . 

The handbooks are readily available to all elements of AMC including person- 
nel and contractors who have a need and/or requirement. The U,S. Army Materiel Com- 
mand policy is to release these Engineering Design Handbooks to other DOD activi- 
ties and their contractors, and other Government agencies, in accordance with 
current Army Regulation 70-31, 9 September 1966. Procedures for acquiring these 
handbooks follow: 

a. Activities within AMC and other DOD agencies should direct their requests 
on an official form to: 

Commanding Officer 
Letterkenny Army Depot 
ATTN: AMXLE-ATD 

Chambersburg, Pennsylvania 1 7 201 
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b. Contractors who have Department of Defense contracts should submit 
their requests, through their contracting officer with proper justification, 
to the address indicated In paragraph a. 

c. Government agencies other than DOD having need for the handbooks 

may submit their request directly to the Letterkenny Army Depot, as indicated 
in paragraph a, or to: 


Commanding General 
U. S. Army Materiel Command 
ATTN : AMCAM-AB5 
Washington, D. C. 20315 

d. Industries not having a Government contract (this includes Universities) 
must forward their requests to: 

Commanding General 
U. S. Army Materiel Command 
ATTN: AMCRD-TV 
Washington, D. C. 20315 

e. All foreign requests must be submitted through the Washington, D. C. 
Embassy to: 

Assistant Chief of Staff for Intelligence 
Foreign Liaison Office 
Department of the Army 
Washington, D. C. 20310 

All requests, other than those originating within the DOD, must be accom- 
panied by a valid justification. 

Comments and suggestions on this handbook are welcome and should be 
addressed to Army Research Of f ice-Durham, Box CM, Duke Station, Durham, North 
Carolina 27706, 



AMCP 708-191 


CHAPTER f 

INTRODUCTION 


1.1 KFINITIONS QF SYSHNS ANALYSIS 

Generally speaking, the nomenclature of Systems Analysis can be applied to 
any systematic approach that compares alternate means of attaining a specified 
objective. The specific techniques and methodologies may differ depending bn 
the many factors of each studyj those inherent due to the class of problem 
and those imputed because of problem variation from a "classic case". However, 
all of the genorlc classes of Systems Analysis studies have the common feature 
of systematically examining all classes of problems, whether simple or complex. 
The application of System Analysis processes are directed towards supplying 
the decision-makers with maximum information, quantified when possible, in 
order to help them in selectirg preferred alternatives to the attainment of 
the stated objective. Also, when no alternative means are clearly visible, 
the process is capable of imparting cogent information which can be utilised in 
the formulation of new alternatives. 

The concept of Systems Analysis has received considerable attention through- 
out Department of Defense areas of interest; Army, Navy, Marine Corp, and Air 
Force. However, the subject and applicability are not exclusively military 
oriented. Extensive use of Systems Analysis has been utilized by non-military 
activities, both in-house governmental agencies as well as the private sector 
of the economy. 

Materiel Systems Analysis has been defined by the United States Army 
Materiel Command as follows: 1 

1. Materiel Systems Analysis - A generic term which implies both a 
' technique and a function which, for the purposes of this regulation, 
are defined as follows: 

a. As a technique — involves the analytic Investigation and quantita- 
tive appraisal and comparison of materiel programs or courses of action in 
terms of the effectiveness, improvement coefficient or cost benefit expected 
versus the costs either required or anticipated to be incurred. Generally, 

*AMCR i l—l ; Research and Development Materiel Systems Analysis; U. S. Army 
Materiel Command, Headquarters, Washington, D. C., 21 April 1970. 
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fof Systems Analysis for materiel Items or programs, the benefits and costs | 

of concern are considered on a "life cycle" basis. Systems Analysis, as -a.# 

a technique, may be applied at any point in the life cycle. 

b. As a function -- Involves the staff and operation activity necessary 
and required to discharge the AMC requirement and responsibilities for Sys- 
tems Analysis In an organized fashion and to fix responsibility. In general, 
the conduct of the Systems Analysis function takes the form of studies, 
projects, and Investigations involving the technique described above and 
the application of modern analytics and' costing procedures. The studies, 
projects, and investigations comprising the function of Systems Analysis 
may variously taka the form and title of cost-effectiveness, parametric 
deslgn/cost-effectiveness (PD/CE), cost-benefit, cost and performance, 
trade-off, optimum mix* and quantitative inventory mix studies and analy- 
ses; product -improvement determinations; and qualitative assessments of 
approaches In functional activities and programs. The techniques of Sys- 
tems Analysis, are equally applicable to all of the above. As o function. 

Systems Analysis seeks to aid the decision making process throughout the 
life cycle of materiel programs. 

1.2 SEFIMITIOM Of MIHFFECTIVINISS 

1.2.1 Cost-Effectiveness Analysis (study) has been defined by the United States 

Army Materiel Command as follows":"^ 

" Cost-effectiveness analysis (study) - The process of comparing alternative 
solutions to mission requirements in terms of value received (effective- 
ness) for the resources expended (costs)" 

1.2.2 Cost-effcctlveness, (C-Ei In generic usage , is interpreted as a measure 

defined implicitly or explicitly by a decision-maker of the benefits to be 

■7 

derived from and the resources expended on a system . J 
This can be functionally expressed as: 

C-E *■ f (benefits derived; resources expended) 

t.S MCKMOtfM AND HISTORY OF SYSTEMS ANALYSIS AND COST-EFFECTIVENESS 

1.3.1 History of Systems Analysis 

Present day use of the word "Systems Analysis" is varied, depending on the 

user. The chtonology of Its constituent elements could (at least) regress tot 


^Maltese, Jasper; ARIHC Research Monograph No. 12., System Cost-Effectiveness; 
Basic Concepts and Framework for Analysis - ARIHC Research Corp. , Annapolis, 
Md., January 19^7; p.9- 


l -? 



AMCP 706*191 


Arl.atoteli.an logic; then to the formulation of method* and procedure* of science 
during the Renaissance (14th-'17th centuries); Fredrick W. Taylor** inception of 
Scientific Management;^ sporadic use of statistical decision making in certain 
World War I studies and introduction of a scientific method consisting of 
objectives, constraints, configuration, selection. Implementation, evaluation, 
feedback and conclusion - known by college students for years as a format for 
problem solving. 

The nearest historical milestone (within the generic context of Systems 
Analysis) that has major import to the ultimate definition Is the development 
and use of operations research in Great Britain during World War II. These 
operations research studies were devoted to early warning systems, anti-aircraft 
gunnery, anti-submarine warfare, civilian defense and conduct of bombing raids. 

A group consisting of Professor P,M,S. Blockett, thiee physiologists, two 
mathematical physicists, one astro physicist, an Army officer, one surveyor, 
a general physicist, and two mathematicians utilized the mixed-group approach 
in solving operational problems.^ This philosophy is certainly Inherent in 
what we now call Systems Analysis, with the inter-disciplinary group being 
necessitated by both the complexity of the problem and its means of solution. 

The main difficulty in describing what "Systems Analysis" is and is not 
can be gleaned from the newly developed classifications of analytical activities 
which have emerged, namely; operations analysis, operations research, systems 
research, systems engineering, cost-effectiveness and management science. 

It is most difficult to determine what the exact definition of each is and 

* 

which one of the subject titles subsumes the others. 

N’aylor, F. w.. Scientific Management; Harper Sc Bros., New York, 19^7 

-’Trefethen, F. N., Operations Research for Management; The John Hopkins Press, 
Daltimore, Md., 195** 

^Fiagle, C., et.al., Operations Research Sc Systems Engineering; The John Hopkins 
Press, Baltimore, Md., I960, p.19. 

In order to explore areas of difference in understanding about Operations 
Research and Systems Engineering activities, it would be well for the reader 
to refer to the following books and periodicals: 

Bronowski, J.; Scientific American, Vol. 185, October 1951# PP.75-77. 

Machol, R.E.; Mechanical Engineering, Vol. 79, No. 9, September 1957, 
pp. 890-91. 

Flagle, C., et.al.; Operations Research * Systems Engineering.: The John 
Hopkins Press, Baltimore, Md., i960, p.19. 

Hall, A. D.; A Methodology for Systems Qigineerlng; D. Van Nostrand Co., 
Inc., Princeton, New Jersey, 1962. 
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After World War II, the RAND Corporation interpreted weapons systems analy- 
sis as a description of those studies which did not have clearly deflnod inputs 
for given objectives and whose future uncertainties were recognised to be less 
well defined than those of earlier studies. 

The post war studies in weapons systems analysis by RAND and other companies 
is the genesis of the term Systems Analysis. Chcrles Hitch, formerly of RAND, 
became Assistant Secretary of Defense, Comptroller 1.. 1961 and introduced the 
concept of Systems Analysis within the Department of Defense. 

Since 1961, the term Systems Analysis has teen used by DoD to describe both 
the philosophy end some of the techniques and methodology applicable to defense 
programming and budgeting. 

In Analysis for Military Decisions , E. S, Quade describes System Analysis, 

"While it does make use of much of the same mathematics (as operations 
research) - it is associated with that clas3 of problems where the diffi- 
culty lies in deciding what ought to be done - not simply how to do it. 

The total analysis is thus likely to be a more complex and less neat 
and tidy procedure, one seldom suitable for quantitative optimization. 

In fact, the process is to a large extent synthesis: the environment will 
have to be forecast, the alternatives designed and the operational laws 
invented. Thus with a systems analysis, one associates ''broad", "long 
range", "high level", "cholce-of-objectives", problems and "choice of 

7 

strategy", "qualitative judgment" and "Assistance to logical thinking". 

In a later definition, E. S, Quade states: 

"System Analysis - a systematic approach to helping a decision-maker 
choose a course of action by Investigating his full problem, searching 
out objectives and alternatives and comparing them in light of their 
consequences, using an appropriate framework - insofar as possible, 
analytic - to bring expert Judgment and intuition to bear on the prob- 
i .«.- 8 

This latter more explicit view of System Analysis seems to be necesbary 
in view of the increasing sophistication of technical programs and studies 
which continually cause the decislon-maker(s) to need more capacity for under- 
standing and recommending the "best approach" The nature of systems analysis 
and its objectives are aimed towards this goal. 

Most of the material presented above relates to the history and interpre- 
tation of systems analysis as viewed by DoD. 

7 

Quade, E. S., Analysis for Military Decisions; Defense Documentation Center, 
Alexandria, Va., AD ^53987, November 196k F p.Y. 

^Originally appeared in the book SYSTEMS ANALYSIS AND POLICY PLANNING, Applica- 
tions In Defense, Edited by E. S. Quade and W. I, Boucher; Published In 1«69 
by American Elsevier Publishing Company. Tnc. 
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Nonmilitary usa of Systems Analysis has, today, culminated In development 
of management information systems. These MIS are the final output of the efforts 
of Systems Analysis with the same stipulated objective as DoD programming - 
that cf providing the maximum cogent information to a decision-maker for e given 
purpose. 

Nonmilitary organizations, generally, do not have the inherent comple. .ty 
of determining the optimum solution to a national defense rosture for a given 
time period, but do have relatively high order of complex problems in such 
areas as space, management science, planning and forecasting, resources manage- 
ment, product line mixes, transportation, communications and participation In 
social welfare programs. 

As can readily be seen, the problems are somewhat similar in total objec- 
tive — The best decision. However, some of the factors aiding utility in 
civilian Systems Analysis are: costs are more readily determinable} competition 
aspects are more quantified and the technology is at hand (or can be determined 
within closer limits than can that of the military). 

The stated aim of materiel Systems Analysis is to insure that the Army 
can accomplish its mission within the level of effectiveness specified and with 
the minimum expenditure of resources. 

This goal encompasses resource management; and although costs have been 
Implied in the foregoing discussion, it now becomes necessary to determine how 
they were derived and how they interface with Systems Analysis. 

1.3*2 History of Cost. Effectiveness 

Throughout history, man has reckoned with U.e cos’, of the item he acquired. 
Somehow, through mutual agreement, or other philosophy, man decided what the 
payment (cost) should bo for what he received. 

Early Greek philosophy gave us the word Economics - then defined as house- 
hold management - which, today, is designated as the branch of social science 
dealing with the description and analysis of the production, distribution and 
consumption of goods and services. 

Economic philosophy started with tne "philosophises school^’ Arif totle 
and St. Thomas Acqulnas (comprehensive codification of "Just-price"), then 
to the modem age economists and the "classical school" (Adam Smith, John 
Stuart Mill, et.al.), then the "Utopian Socialists" (Robert Owen) and "Scien- 
tific Socialists" (Karl Marx). 
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TcJ-i/ r,v>iit economic W eory 1 b classified ae being neo-classical iynthllil| 
a .’.arrlugn between Bile ro and macroeconomics . 

Tint ••ssenei- of *neu-c Ian. leal synthesis la the modern day Interpretation 

Q 

of e< onumlo analysis. In terns of J. M. Keynes: 

"The object of our unulysiu la not to provide a machine or method of 
blind manipulation which will furnish ai infallible answer, but to pro- 
vide ourselves with an organized and orderly method of thinking out our 
particular problems; and, after we have reached a provisional conclusion 
by Isolating the complicating factors one by one, we then have to go 
back on ourselves anci allow, as well as we can, for the probable inter- 
actions of the factors amongst, themselves. This is the nature of 
economic thinking." 

Interpretation of the above-mentioned cconomi 1 - philosophy certainly shows 
the genesis of modern analytical thought that is now embodied in the definition 
of Systems Analysis and/or Cost-Effectiveness. 

When this philosophy is combined with the theo y of Production; Theory of 
Input-OUtput Analysis (see also linear programming); Economic Welfare Theory; 
and such economic s-oriented definitions at; Cost (Resources); Goods and Services 
Value, Price and Utility, (see also marginal utility), it becomes apparent that 
within the concept of r,eo-class leal synthesis lies the springboard from our 
definition of Cost-Effectiveness. 

More specifically. Economic Welfare Theory constituents of positive 
theory and welfare theory describe the evolution as such: 

Positive theo ry considers the development of economic principles of 
operation regardless of desirability or not, 

Welfare theor y is concerned with, the evaluation of the operation of 
economy ,'.n terms of assumed standards. 

Tne overall objective of Welfare Economics if stated in the term Benefit-Cost 
Analysis: A means cf estimating the prospective economic returns of a pro- 
ject (or projects) J.n ’•elation to costs. 

Comparison of B enefit-Cost, Analysis and Co3t.-Ef fectlveness leaves little ^ 
doubt an to the specific genesis of the term Cost-Effect, veness or Cost- 
Effectiveness Analysis as defined herein. 

The evolution of the term "Cost-Effectiveness Analysis" occurred after 
World War II (see paragraph 1.3*1 under Systems Analysis). Cost-Effectiveness, 
per :ie, appears to have been formally Introduced during the period from 1961 

■'Keynes . -T. M . } General Theory of Employment, Interest, and Money; Harcourt, 
Brace; flew York, 1936, p.267. 
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to 1964. This la evidenced by Inclusion of coi»t-effectlvene«s requirement* 
formully stipulated In certain type Request for Proposal development/procure- 
ments contracting efforts In accordance with DOD Directive (Series 3900*9; 

1964) and The Contract Formulation - Contract Definition Concept Programing 
of the DoD during this and ensuing periods of time. 

1.3.3 Systems Analyels/Coat-Effectlveness 

ThP literal combining of the terms Systems Analysis and Cost-Effectiveness, 

In view of their previously developed history and subsequent definition could, 
upon examination, raise much doubt about what each does that the other doesn't =. 

Immediate questions are: 

Can they be combined as SA/CE?, 

What do they each mean in this form?, 

What methodology combines them?, and 
Aren't they interpretatively redundant? 

Previous history and definition of Systems Analysis Illustrates that it 
is more likely to deal with that class of problems directed towards what should 
be done, not the methodology of how to do it. In this sense, then, it 1 b di- 
rected at the suitability of Implementing a specific method and the consideration 
of alternatives directed towards the implementation of this method. 

When the effort is directed towards the costs (and/or resources) required 
between these alternatives, and the effect of changes in either cost or effec- 
tiveness, relative to each other and mission objectives, then we use the term 
cost-effectiveness analysis. 

The objective of cost-effectiveness is, usually, to minimize the costs 
(resources) at which a given level of effectiveness can be attained for a given 
mission. This also includes the various supporting functions. 

In order to further clarify the specific definition of each term and to 
illustrate their integration, the reader is referred to the immediate following 
sections, 1.4 (Methodology of SA/CE) and 1.5 (Application of SA/CE) . Also, 
the subsequent chapters of this Guidebook are directed towards defining the 
role of Systems Analysis and Cost-Ef fectivenese aa they are considered in the 
ensuing analytical formats. 

1.4 MEYNMttMY If SVSYtMS MUlYSIS/COSMFFCCTITimH 

There isn't any singular formulation nor is there a standard methodology 
which is applicable "across-the-board" that allows Systems Analysis/Cost- 
Effect ivancr* studies to be performed tor all classes or sub-classes of problems 
explicitly. 
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The Immediate lack of an analytical "cookbook" approach doean't preclude the 
implementation of Systems Analysis/Coat -Effectiveness studies however. The main 
virtue of any stipulated scientific method or programming function is its recog- 
nition of chaise. Comparison of a generalised "x-atep" scientific approach with 
the Systems Analys is/Cosc-Effectlvenese methodology presented herein reveals 
both to be dynamic, adaptive processes. The singular discrete difference is in 
problem formulation and solving activities, due primarily to differences In 
levels of abstraction. 

In order to perform any analysis it is necessary to conceive a disciplined 
framework, i.e., a systematic approach, with provisions for making comparisons 
between alternative ways of accomplishing an objective systematically (hopefully 
quantitatively) in a logical format that can be retraced and verified. 

Systems Analysis and Cost-Effectiveness studies utilize the same basic 
framework for their objectives; therefore, it r.ow becomes necessary to differ- 
entiate between them in terms of the definitions given in sections 1,3.1 and 
1.3.2. The main difference appears to be in the degree of applied emphasis. 

When the study is directed towards the determination of "costs' between similar 
systems that can attain a specific objective, the term Cost-Effectiveness analy- 
sis is applied. When the problem is one of broader scope; i.e., consideration 
of different types of systems that could attain the specific objective, then the 
term Systems Analysis is used. 

Decisions pertaining to choices of alternative weapon systems or force struc- 
tures and the strategies for their employment are essentially matters of economic 
choice. Certain elements have evolved which are common to these kinds of deci- 
sions and have been contained in Systems Analysis/Cost-Sffectlvencss studies. 10 

1. Objective - Systems Analysis/Cost-Effectlveness studies are initiated 
In order to aid In determining a particular policy and/or procedure. These 
analyses are directed toward a description of the objectives - what they 
should be (or are). This done, the various policies and procedures are 
evaluated, compared and "scaled" in order to determine what their 
effectiveness and costs are and to what degree they do "ntts'n" the 
objectlve(s) . 

2. Alternatives - These are the various means that can be used to attain 
the objective. The alternatives presented- should Include all kn^wn methods 
(also, within a given time frame, consideration of new means within the 

"then" known state-of-the-art) that can achieve the desired results. 

The alternatives can be not quite obvious and consideration of all types 
and ways of doing things must be Included. (As an example, If the ob- 
jective of any given period of history was pence - one philosophy was 
to negotiate - another was war. War was a means of attaining peace 
by unification.) 

‘ Q 0uade, E. S., Analysis for Military Decisions; AD p.’ r *,. 
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3« Costs - The costs are, also, not. readily obvious. These are the aua 
total of the resources expended to attain the objeotive(a) for each of 
the proposed alternatives. Resource* (coeta) are those items consumed 
In the attainment of the stipulated objective which cannot be used for 
other purposes . Total costs must include consideration of ail the 
factors Involved In accomplishing an objective. (As an example, the 
cost of smoking Is determinable at "y" cents per pack. However, If 
the U. S. Surgeon General's Report Is correct, In that smoking reduces 
life by (on the average) 8.3 years, then the value of reduced life 
is a cost to be determined and applied to the sum total.) 

4. Model - The model is a representation of the reality of the situa- 
tion or condition being studied. Ideally, It would represent the real 
situation without error or uncertainty. Usually, In Systems Analysis/ 
Cost Effectiveness studies it can simulate (at best) most, or eome por- 
tion, of the real world. The model defines its representation of the 
real world, and through various exercises, simulations, gaming and 
mathematical representations, supplies numerics or information on the 
effectiveness of the various alternatives under consideration for 
in attainment of objectives. 

The structure and capability of the model Is a major limiting fea- 
ture of an ana.lysls. 

A basic requirement of any model is that it should provide correct 
answere to the stated questions in an economical manner. Tills causes 
consideration of the following factors: representativeness, uncertainty, 
data sources and validity; i.e., consistency, sensitivity, plausibility, 
criticality, workability, and suitability. 

5* Criteria - The criteria are the standards or accepted rules which 
are used to determine the relativeness or desirability of one alterna- 
tive vs. another and allows for choosing one in preference to others. 

In a Cost-Effectiveness analysis it provides for weighing and combining 
cost vs. effectiveness. 

At this point in Systems Analysis/Cost-Sffectlveness studies we 
can now Interrogate the major operating elements of the scientific 
methodology within the general framework of the processes of a system- 
atic analysis. Regardless which procedure of scientific inquiry is 
Invoked, the analysis proceeds through the following typical stages: 

Hypothesis - At this step, the objectlve(s) are defined. The 
constituent elements Identified, and the extent of the problem 
delimited to suit knowledge, time and cost considerations. 

Definition - This step explores the alternate configurations, 
policies or programs that can be directed towards solution of the 
problem (objective). Inherent In this step are considerations of 
the resources and other Interrelationships. 
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Analysis - Construction of the model (s) at the necessary level of 
abstraction; and exercising the model to determine the consequences 
of alternate program* and considered factors, i . 

Evaluation - This step examines the derived results. It is here 
that the preferred alternative is identified. The evaluation Is 
based on modification of all factors discovered in the iterative 
analysis. 

Conclusion - This step Is concerned with the verification of the 
resultant analysis by test and/or experimentation. 

The above application of the systematic approach is not new and is quite 
stralgnt forward. The problems encountered in using the approach are not caused 
by its non-applicability, but because of the /ugariea of the environment we are 
attempting to use it in and for. 

In Systems Analysls/Cost-Eff ectlveneos studies the decision-maker end 
analysts would like to live and operate in a deterministic atmosphere, and, 
at worst, in a well behaved s' ochastic environment. In this environment, the 
decislon-maicers/analysts car. modify hypotheses, which are " subjectively probabi- 
listic at worst, with information gleaned from application of a defined scien- 
tific methodology, causing revision of the original probabilistic function 
towards a fully descriptive and validated function which expresses the actual 
environment exactly. 

Unfortunately, the real world precludes such a standardized and Ideal 
approach and solution to problems encountered by present day Systems Analysis/ 
Cost-Effectiveness studies. 

Consider the objective of military posture at seme time in the future. What 
3hould its composition be and what should it be capable of? At once, the objec- 
tives a>-e not even known, are certainly multiple and what can be postulated as 
a figure of merit? Knowing that the Systems Analysis function would be to con- 
sider not only pure military posture, but with consideration of the interrela- 
tionships between socio-economic and political factors and military affairs, 
it becomes immediately clear that a "model" would be quite difficult to con- 
struct . 

The analytical framework postulated above ennnot accomplish analyses such 

as lr, necessary in view of these objectives in a single pass-through. In order 

\ . . 

to maximize the amount of information that can be obtained from such an ap- 
proach, It is necessary to Iteratively exercise the framework. Selection of 
objectives, alternatives , data collection, modelling activities, establishing 
measures of effectiveness and figures of merit, sensitivity analyses, evalua- 
tions, modifications and conclusions should be Iterated through the established 
analytical framework In an attempt to remove the "Impurities" of the first 
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run-throu&ji and reduce the overall complex subjective area Into a quantifiable 
real world area an near as can be accomplished. 

The Iterations produce: .ellmlted, redefined or changed objt -tlves (per- 
haps even causing suboptimization to be considered in view of total complexity); 
discovery of new alternatives as by-products of analysis, or modification of 
type and quantities; redetermination of costr or resources due perhaps to con- 
sideration of having to consider non-dollar cost contingencies; necessity of 
modelling changes in order tc more accurately depl t the "real world" consider- 
ing, also, the constraints ana configurations In terms of new data or redefini- 
tion of objectives and measures of effectiveness and establishment of new 
criteria in terms of new data, information and/or changer in the Inherent ele- 
ments in the analysis or in more accurate standards or ruler sc (delineated by 
recursion. 

Obvioudly, the process can continue indefinitely; t normally. It It 
exercised until the results are deemed satisfactory or the constraints of time 
and/or money force discontinuance. 

At this point In the dessertation it is necessary to digress from the prime 
purpose of this section - Methodology of Systems Analysis/Cost-Effectiveness - 
which explains the how It la done, and ask why and for whom. Subsequent sec- 
tions explain what and when as well as modification of how and why; but, for 
whom needs a brief explanatory section. 

This entire SA/CE studies-snalysis is directed towards improving the quan- 
tity, quality, and accuracy of information that is necessary (or would be help- 
ful) for a decision-maker, in order that he may make the best possible decision 
with minimum uncertainty and risk - or, if you will - the decision as to what 
is more effective, economical and timely. As such, SA/CE 3s a prime mer.igement 
tool, nothing more - but nothing less. 

Granted, the genesis of the reason or need for a study 13 based on specu- 
lation of "decision-makers" and that it does contain subjective value Judgments, 
lack of precise knowledge and/or data, urcertalnty of competitive rtrategy and 
other uncertainties. However, the studies car. aid the decision -makers by 
assessment of implications gleaned from selection of various alternatives. 

Systems Analysis concerns itself with problems of resource allocation, i.e , 
wnat mix of "things" should be obtained and how Ion* art they to be considered 
adequate for their purpose. Cost-Ef fectlvenese reveals the cost versus effec- 
tiveness of the "mix" and aids in determination of whet ir the best way to 
"spend" the ’•oeourceo. 
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Without programming this philosophy through a systematic analytical format, 
there Is no better way- of determining how many of what can accomplish a mission - 
i.e., - it -Is not sensible to intuitively prescribe a posture without considering 
whatever numerics car. be gained from a "study" and what their information content 
is. It is not the purpose, h*ir can it be accomplished by SA/CE, to cause a 
decision-maker to agree leyond all>.doubt that this information does indeed pre- 
sent the course of action to follow. 

The prime purpose is to provide o.s much quantified information - including 
limiting conditions whether.truly quantified or qualitative - as is possible 
in order to sharpen the intuition of the decis'.cn-makers sued help them arrive 
at the oest solution in terms of their observation, intuition, experience and 
value judgments. 

Discussion of limitations of SA/CE will be presented in the final segment 
of this chapter. ^ 

1.S APPliSATISA Or SYSTEMS ANALYSIS/COST -EFFECTVEAESi 

1-5- i General 

The concepts and philosophy of Systems Analysis/Ccst-Effectiveness can be 
applied to almost an;.’ system at any time during the life cycle/' 

The Ktrmy Materiel Command (AMC) states that a requirement for a Systems 
Analysis/Cost -Effectiveness study, evaluation, or investigation m&J/ exist or 
Oce initiated in support' of concept formulation (CP), contract definition (CD), 
program, change requests (PCR's), program submissions, technical feasibility 
studies (PS's), alW studies associated with qualitative material development 
objectives (QMDO's), qualitative material approaches (SjMA's), advanced develop- 
ment objectives /ADO' s) , advanced development plans (ADP's) and technical 
development plans (TCP's).* in addition,, other phases m the life cycle where 
tnis type rf evaluation should be applied are in long range R & D systems pla n- 
n.rg, maintainability and reliability studies, us well, --s in major inventory 
and log i.? tics decisions. 

The study may take on any of the forms in Figure 1-1, depending upon where 
1.. Is in the life cycle as well as what type cf decision must be made. 


*\MCR 11-1, 21 April 1970. 
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(1) 

System-Effectiveness Studies 



* Investigate s technical feasibility 


• Predicts how well a system will meet mission requirements 

(2> 

System -Analysis Studies 



• Technical -feasibility studies and concept selection j 


• Effectiveness Analysis 

► Inter-3ystem trade-offs 


• Cost Analysis 

(3) 

Cost-Effectiveness Studies 



■ Effectiveness Analysis 

► Intra -System trade-offs 


• Cost Analysis 


FKVtt 1-1 

TYPES IF SYSTfMSMALYStt STIIIES 


Category 

Orientation 

SA/CE Milestones 

Research (6.. 11) 

Increase knowledge 


Exploratory Development (6.21) 

Technical feasibility 

Technical feasibility and 
concept selection studies 

Advanced Development (6.31) 

Operational and tech- 
nical suitability 

Inter-system C-E studies 

Engineering Development (6.4l) 

Design Engineering 

Intra-system PD/CE studies 

Operational Systems Develop- 
ment (6.71) 

Production Engineering 



F ISSUE 1-2 
MTU CYCLE 
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Ttw relationship of the types of studies to the major phases in the RDT * E 
Cyo . c is contained in Figure 1-2. During exploratory development (6.?1) the f j 

type of system analysis studies that are conducted are normally concerned with 
technical feasibility and concept selection. However, when advanced develop- 
ment (6. 31) begins, the primary aim is to conduct Inter-system traae-offs In 
order to choose among several alternative systems capable of performing some 
given mission, assuming that all contending systems are capable of perfcj mar.-.- 
at various levels of effectiveness. 

In engineering development (6.41) parametric design/cost effectiveness 
(PLVC’E) studies are conducted. This is a process of formulating and evaluating 
a complete range of alternative Intrasysten trade-offs of components fi.e., 
designs) to provide the optimum capability for fulfilling a sivcr. v-i-m 

1 . £ 3 . 2 Application of 3A/CE to Concept Formulation and Contract Definition 

Recent DoD and Arvy Directives* which establish the concept formulation and 
contract definition phases of the system life cycle show an Increasing aware- 
mss of and need for Army program managers to make 3C>und decisions based upon 
quantitative evaluations which should result in economical and operationally 
effective system designs capable of meeting the desired performance require- 
ments. In this section, concept formulation and contract definition will be 
described and the requirements for Systems Analyeis/CoFt-Effectlveness stucle3 
will be discussed. 

The objective of concept formulation is to pre/ide the technical, economic, 
and military basis for a conditional decision to initiate engineering develop- 
ment. 


It Is accomplished through comprehensive system studies in exploratory and 
advanced development by means of experimental tests, engineering and analytical 
studies. This work constitutes the necessary preliminary threat and operational 
analyses, trade-off and cost effectiveness studies, and development of components 
and technology - to assure a firm foundation for Engineering Development. The 
evidence required for a conditional decision to proceed with Engineering Develop- 
ment includes the following prerequisites: 

(a) Primarily engineering rather than experimental .effort Is 
required, and the technology needed In sufficiently in hand. 

(b) The mission and performance envelopes are defined. 

(c) The best technical approaches have been selected. 

(d) A thorough tredo-off analysis has been made. 

•TOT TiTrecTTve 32f/, . 0 . AR 70D-D. 
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(e) The coot •effectiveness of the proposed item Is favorable 
In relation to that of competing Items on a defense-wide 
basis. 

(f) Cost and schedule estimates are credible and acceptable. 

On the basis of this information, the Army requests approval, to initiate 
Engineering Devel cipment . The request Is made either by memorandum to DDR Sc E, 
c r If require' 1 , by a Program Change Request (PCR). It is accompanied by a 
^ethnical Development Plan (TDT), specifically addressed to the six prerequi- 
sites cited above, which summarises pertinent cost-effectiveness studies and 
developments and provides whatever information may be required to substantiate 
the achievement of these prerequisites. 

If the initiation of Engineering Development receives conditional approval, 
the Contract Definition phase begins. The objective of Contract Definition is 
to determine whether the conditional decision to proceed with Engineering 
Development should be ratified. Its ultimate goal is achievable, firm and 
realistic performance specifications, backed by a firm fixed price or fully 
structured Incentive proposal for Engineering Development, in addition, It 
embraces the following subsidiary objectives: 

(a) Precisely define Interfaces and r isponslbilities . 

(b) Identify high risk areas. 

(c) Verify technical approaches. 

(d) Establish firm, realistic schedules and cost estimates t'er 
Engineering Development, including production engineering, 
facilities, construction, and production hardware that will 
be funded during Engineering Development because of con- 
currency consideration. 

(e) Establish schedules and cost estimates for planning purposes 
for the total project, including production, operation, and 
maintenance. 

The Contract Definition procedure is mandatory for all new Engineering 
Developments or Operational Systems Developments (or major modifications of 
existing ones) that are estimated to require total cumulative research, develop- 
ment, test, and evaluation financing in excess of $25 million, or a total pro- 
duction investment In exoes3 of $100 million. (However, DoD, DA, or AMC may 
require Contract Definition on other syetems which are below the $25 million 
and $100 million threshholds. ) 

Contract Definition is normally performed by two or more contractors in 
competition under the technical direction of the cognizant Army activity. 

It may, however, be performed by a sole source contractor if necessary. 
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The trade-off studies that are conducted during thin phase should be directed 
toward achieving an optimum balance between total coBt, schedule, and opera- 
tional effectiveness for the system. Total cost (or life cycle coat) Includes 
the cost of development, produo*i.Ai, deployment, operation, and maintenance. 
Operational effectiveness includes all the factors that influence effectiveness 
in operational use, as well as inherent or pure performance characteristics 
(as In WSEIAC, the ADC matrices). 

Tlit; system Includes not only the hardwaie but also all other required Items, 
inch as facilities, data, training equipment, and the operational and support 
personnel who will be required. 

The end product of Contract Definition is a complete technical, manage- 
ment, and cost proposal package for Engineering Development. The contractor's 
package shouxd Include such information as a list of the end items required; 
performance specifications for each item; a work breakdown structure and a 
PERT network plan; the principal objectives and features of the overall system 
design, including recommendations for it3 operational use; a recommended main- 
tenance plan: detailed cost estimates and milestone schedules for five years 
beyond It; quantitative reliability and maintainable ity specifications and 
tost plans; time/ccst performance trade-off decisions on major alternatives; 
required new designs and technology; foreseeable technical problems and pro- 
posed solutions; technical specifications and performance specifications for 
supcort items (facilities, training devices, and so on) for which early Engineer- 
ing Development is required; delivery scheduler, and requirements for data and 
documentation; and a proposed schedule of production engineering and production 
tooling in relation to Engineering Development, if appropriate. 

After a review of the contractors' Contract Definition proposal packages, 
the Army recommends one of the following actions: to contract for Engineering 
Development on the basis of the proposals received; to contract with an alter- 
native source; to continue further Contract Definition effort; to defer or 
abandon the Engineering Development effort; or to undertake further Exploratory 
or Advanced Development of key components and/or systems studies. 

1 . R . 3 Conclusion 

The methodology and application of Systems Analysis/Cost-Effectlveness 
studies is directed toward supplying the decision-maker with the maximum amount, 
of quantifiable information about alternative approaches to attaining a mission. 
Also, it stipulates areas of qualified considerations with this, and allows the 
virtues of a systematic approach - the design and development of proposed objec- 
tives and their solutions within a rigorous, logical, adaptive, dynamic framework 
which can be retraced and verified. 
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Thera are many advantages to Systems Analyaia and Cost-Effectiveness. How- 
ever, th y are not panaceas that handle all problems of the Byatem developer, 
manager, or user, nor are they without limitations. Systems Analysls/Cost- 
Effecttveneoe studies must be examined to recognize the limitations built into 
them, or the premises generated based on given information. 

The more prominent limitations Inherent in all but extremely simple analyti- 
cal studies are as follows; 

' Inadequate Problem Definition 
' Improperly Defined Scope 
* Restriction of Alternatives 


' Improper Criteria 

* Interjection of Bias 
‘ Improper Data Usage 

’ Incompatible Moccl 
' Misapplication of Model 

* Forcing Problem into Improper Framework 

* Improper Handling of Relevant Factors 

* Poor Assumptions 

’ Ignoring Uncertainties 
' Misinterpreting Model Results 
‘ Insufficient Samples 

* Failure to Reappraise 

* Failure to Communicate 

Measures of Effectiveness Approximate 

* Future Uncertainties 

' Analysis Never Truly Complete 

* Changing Value Systems 
Neglect of Subjective Elements 

* Assignment of Value to "Costs" (Economic Costy) 
Inability to Verify Declblon 
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Consl deration or each element described as a limitation la not the purpose 
or tula section. However, In order to show relevancy of limitations and tne 
dangers associated with their non-consideration, a few examples will be given. 

First, consider the list of limiting elements dercribevi above. If an 
analysis can never be complete, due to the state of knowledge, time and money. 
It is fair to assume that this list of elements is not complete. 

Consider, also, the statement, "Mustang makes it happen* as an aid to de- 
cision making about buying a car. If I am a car buyer, I must make a decision 
based on what the statement means to me. 

(a) I buy a Mustang and wait for JLt to happen and, unless 
I am purely adventurous, I assume JLt Is good. 

(b) I don 1 t buy a Mustang because, if JJL happens, T am 
in no position to deal with it; or _it Is bad (In my 
opinion) and I don't want _it to happen. 

How, either of these decisions can be a misinterpretation depending on the 
"true" significance of it. 

The real difficulty Is in the word (information) it; what problem did It 
answer? The same is true with results of analyses. Do they literally answer 
the problem or do I still have to Interpret meaning within ny bias? 

As another example; interpretation of a real weather report which states - 
"Probability of rain - 9056 in 25# of the area 10# of the time". I woui. ‘ assume 
that there Is * good chance of rain; that 25# of the area will get rain; and 
of the duration of time (which this event encompasses) it will rain 10# of that 
period of time. 

How if the problem is where do I go so that I won't get wet, how do I 
interpret this? It would appear I would, generally, be wrong in whatever I 
decide, or have minimum confidence that my decision is correct. 

This example, also Considers dangers from the elements of limitation of 
poor assumptions. Improperly defined "model" and failure to communicate, at 
least. Analogous to thi3 problem is the one of forecasting advanced weaponry 
for us-; In, say, 19-85. What, how many, why and against whom? Basically, the 
hard information comes from back-feed and is tempered by intuition and Judgment 
to transcend now to the future. However, until the "systems" are operational, 
in their then environment and we car. obtain feedback, we will always have error 
terms in the analysis - until we pass ..hrough the verif ieatior/modif icaticn 
stages. 
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Consider, also, data errors and modelling problems In determining how to 
attach significance to deterrence. Actually, deterrence is, mostly, a natter 
of philosophy. What does an enemy consider _it to be - and how do you. find 
out? Also, It will be a changing value with time. 

Quantification of numerics la a problem in the area of ’’costa". How 
accurately can you predict costs for the future - even in dollars - much leas 
recognize a variable depletion of resources - with a changing value system? 

Obviously, it is impossible to consider all the limitations applicable to 
SA/CE analytical functions, an some apply and some do not, depending on the 
complexity of the problems. 

Yet, the recognition of them and the decree of their accountability la a 
by-product of an analysis, and does supply information as much as the hard 
numerics. By the recognition (and measure) of accountability attained, the 
analytical processes become better by degree and help to sharpen the intuition 
and Judgment functions of the decision-maker. 
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CHAPTER 2 

GENERAL METHODOLOGICAL APPROACH TO SYSTEMS ANALYSIS/COST* 

EFFECTIVENESS STUDIES 

The general methodological approach to the system!? analyBl8/cost-efrecti’'e- 
cess process le shown In Figure 2-1. 

The approach shown le applicable to both systems analysis and cost-effer- 
tlveness. The distinction between systems analysis and coat effe -tl venec^ 1c 
mainly a difference In the definition of the scope of the study, .''••'stems 
analysis studies are concerned with problems of large scope and are character- 
ized by the comparison of different types of equipments or systeme to determine 
the beat approach for meeting some stated requirement. Technical feasibility, 
Inter-system trade-offs, and the parameterizing of requirements are all as- 
sociated with systems analysis. Cost-effectiveness studies, on the other hand, 
deal with narrower problems. Normally, In cost-effectiveness stud'es, the type 
of system to be analyzed will be given, and the problem is -o conduct lrtra- 
system trade-ofrs to "optimize" the system, l.e. , to develop the best system 
with respect to performance, coot, schedule, manpower, etc. 

2.1 INPUT INFORMATION 

The Initial Inputs that lead to the development of requlremen* a arid 
objectives In the systems analyolo/cost-effeetlveness process are normally a'- 
trltutable to 3uch documents as the Amy Force Development Plan, the Combat 
Development Objective Guide, a Qualitative Materiel Development Objective, C r 
a Qualitative Materiel Requirement. 

2.1.1 Army Force Development Plan (AFDP) 

The AFDP is a responsibility of the Assistant Chief of Staff for Force 
Development (ACSFOR) and Is cons’- ra Ined by anticipated resource llmi'atiops. 

It provides the planning basis for the Five-Year Force Structure and Financial 
Program, and its objective Is to provluc *he bep* possible Amy pos’ure within 
available resources. Specifically, St accomplishes the following: 

It plans the development of balanced capabilities wl’hln established 
constraints and strives to achieve the best possible balance between 
forces, readiness, and modernization. 
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• tt plans incremental Incx-eaeea In capab*. 1 1* ter In order of priori’ v b* u 

dentines the associated add 1 t loua 1 recc-ureen heccf uni'y *.o attain 
In a reasonub la time. 

• I*; plana inereinentnl decreases In capabill’les In inverse oi’der of 
c r ! * 1 cal t ty to provide resources to men ur-programmed requl i-emen*. s . 

2.1.2 Com) at Development Oh^ec' 1 ves Quid * ( CDUO \ 

The CDOO, prepared by the ’JSAC!?, pr vvldes guidance for Army combat - 
development activities and the research ar.a development program. If contains 
all the DA-upproved opera’: Iona 1 and organ! xa’ lorial objectives, Qualitative 
Materiel Development Objectives, Q ,»a 1 ! ♦ a 1 1 ve Mater’ el Requirements, and Small 
Development Requirements' and ‘heir priori’ leu. jt f, j co porr a Ins a romp*, la t '.os. 
of studies, field experiments, ar.d tea's. These contents are defined as 
follows : 

• Ope rat lotlal objectiv e. An operational objective Is an Aray-approvec ne.-'i 
for a new or Improved opera* lorial sapoblllty the* pertains to opera’ tonal 
concepts, tactics, techniques, and procedures. 

• Organizational object ive. An organisational objective Is an Army -approv- 
ed need for a new or revised organization *o Improve Army operational 
capablllt lea . 

• Qualitative Materiel Development Objective (QKDO) . A QMDO Is an Arcty - 
approved statement of a military need for the developmen* of r.ew materiel, 
the feasibility of which cannot be determined sufficiently to percr.l* the 
establishment of a Qualitative Materiel Requirement (QKR). 

’ Qualitative Materiel Requirement (QMB) . A OMR la an Army-approved state- 
ment of military need for a new Item, system, ci? assemblage, the develop- 
ment of which Is believed feasible. 

• Small Development Requirement (SDR) . Ar. SDR states an Army need for the 
development of equipmen' that can be developed In a relatively short 
time ond doe3 no' warrant the major effort required In developing a QXH. 

An SDR Is normally considered the appropriate requirements document If: 

(1) development of the item Is of proven feasibility: (2) 'he time re- 
quired for development is 2 years or less: and (3) the RDTE costs will 
not exceed $2 . r mlll’on, and the projected investment of PEMA funds will 
not exceed $10 million. 

2.1.3 Qualitative Materiel D eve lopmen* Objective (QMDO) Approval 

The first formal requirements document that the Army ueen in the research 
and development cycle is the QKDO. Any individual (military or civilian), uni*, 
or agency may propose a concept or luea 'hat migh' lead to ec'abllshlng a QKDO. 
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However.- CW It) the authority ft ml rerpons Ibl Jlty for preparation and 

:: fbr.tsjflos! of tiML'O'e *.n ACSPOH l'cr approve . -’DC u»«r Opera! lone -research 
• •» jbnlque? to relate t utvt UiCtios t «> 'he two .'.rival estimate* and concept 0 
f ..rr.1'»he«i by AMO ai? a f'ev.lt of c '>an lc and applied research work. - Evaluation 
i f reqwlrener.ie fot mwer-iei for tt.r nmy dicta tea '.‘on sinuous liaison' between 
.'.'v - t r .-} tie 1«iv« loping .agencies of 0!X; a* the laboratory and commodity command 
'.ever. from, this point on all research and development must be supported, by 
e 1 ‘her an object lveo >r a reqvlrer.entB document such as a QKL'O, QMTt, or .*>Z)R 
•nleaa otherwise directed by appropria* - HA authority. 


The tiMpo 'nomaily beg 1 no v;i • h re-pa lrercnv » genera tec by combe t -development 
at -uites conduced by CPC. These requirements, wtc-r. reviewed, Indicate whether 
they are vrtt.hld ’to state-of-the-art or have been sufficiently exp oited ‘0 per- 
.c.lt ! tuned t ate development. ■’T-.e QMHO Is associated with re qi«! remen* 9 net'-ssl'a* - 


: ru 


.er research. 


.1 point we are s‘111 several years {'*.•*!) away fror 


es'nbllehiog f irtn military 31 aracterls* Jus. The QMDO rr?£ !: general tents 
the objective, the operational ar.d organ'. za - Inna 1 conc’ptc Ter employment, a 
dec trip!: ion of full Justification l'cr new 
Usually, the QKDO is first •1r ;:a'ei 
1 . terested commands for ':o'vt.en‘.s arc. r .rrer ' 
consists of: 


'tern, and its priori*! (1, II, cr 
It deaf foi-r, to AMC and c'her 
d revisions. i-.’AC’s Inpc gei erelly 


• A "ball park" cor i flgu:-* for * ne required research and vxr.'.ora' or;.' 
development 

• A time-frame for eom.ple* of' * ?.«.• work 

• The effect performance m. ! v: ' }/.:••« or. -already apppo'ved prom:.? 

• The potential of present knowledge * t ad van fe the state -of-* nr -art r f- 
ficlently -to make the project feasible 

QMDO's rnay be initiated at any stage in the research-and-developnent 
cycle and may result from a combat-development study, operational experience 


developmental experience, technological breakthroughs, or feedback of defi- 
ciencies in existing equipment. 

The Director of Research, Development, and Engineering, Headquarters, 
AMC, is the AMC focal point for processing all new materiel objectives docu- 
ments and for ensuring that adequate QMD0 plans are prepared. After review- 
ing all the comments from other agencies on the draft QMDO, CDC revises it 
and submits it to DA for formal staffing and approval. ACS r OR has DA staff 
responsibility for all QMDO’s. After DA approval, QMDO's are published in 
CDOG so that all interested commands and agencies can be made aware of the 
object iveg . 



• ••• ^witltatlvt? Material Requirement (QMR) 
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After renearch and exploratory development have pro graoeed to the point 
where AMO fee la It may bo possible to trat elate technlca* knowledge acquired 
by the laboratories Into a feasible system, It will prepare a draft QMS or 
recommend to CDC that u OMR be prepared and Js.ued. The QMS Is directed toward 
attaining new or substantially Imprcved materiel that will significantly advance 
the Army ability to accomplish its mission. Unlike the the CiMH is much 

mere specific In describing the requirement. It states major materiel needs in 
terns of mtlltury characteristics and priorities and relates materiel to the 
operational and organizational context In which it will be used. 


2.2 DEFINE REQ’llfEMENfS AND OBJECTIVES 

The basic task bf systems analysis Is to evaluate systew. In terms of 
achieving objectives and consuming resources. While this evaluation might be 
male at any point in the life cycle of a system, major Interest currently lies 
In making such evaluations before resources are committed to creating a system. 
Major emphasis in thic section is therefore given to such prior evaluations. 

In any systems analysis, it Is essential to take- the information as stated 
in the study directive and define an acceptable set of requirements and 
objectives. Stating the detailed requirements and objectives is a major part 
of the study effort; they may require updating and redefinition following 
evaluation and feedback. 

In establishing the objective, it is necessary to define the boundaries 
of the analysis In terms of system objectives, system definition, associate i 
operations, and influencing factors. Initially, the objective should be stated 
in general terms so that a comprehensive analysis can be made. Extreme .-are 
and effort must be made to properly state the system's objectives. It is 
essential to the entire analysis, since It is the major factor in th*“ selection 
of the missions, the synthesis of alternative systems, the evaluation ■'riteria 
as well as dictating the types of models which must be employed In the study. 

It is not always easy *- 0 decide what lr 10 be Included In the analysis. Often 
the decision must be made subjectively, and 1 * way be heavily influenced by 
such factors as data inadequacies, schedule .and manpower limitations, and 
uncerta lnties as to the appropriate method for model trig particular aspecti < 
the problem, nevertheless proper statement of the et jectivt* must be appror. .*, ? 
as o crl* leal first stop" In the system annlys l:v •ost-.o? feet lveness process- - . 

After obtaining a definitive s t at •■men* of Mr- :y :•?**•?. < ijec! ! V' and > »**rcj ! refer 
,f f r,e system boundaries, * he npee.ifl c tasks or c'.-.ji’r. pro: lies 


are re: crated. 





i7-;-r 


>v. 

JcS&jW ^V Vy-^y*~ *•* >*' — <’• • * 



AMC^ 706*191 
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Tc propel’ ly state the system's objectives , the analyst must rely heavily 
both on past experience gained through conducting and evaluating similar 
studies as well as his knowledge of the military environment and operation. 

There are three pitfalls encountered in specifying system objectives: 

(1) che analyst may state the objectives too broadly, (2) the objectives 
may be limited in scope, or (3) the objectives may be stated in such a man- 
ner, as to describe a particular system rather than a functional need. If the 
objectives are too broad, this results in an analysis which Includes an im- 
pr&ctically large number of alternatives. Additionally, broad objectives often 
lead to a large number of assumptions which must oe made in order to evaluate 
the system(s). 

Generally, as the number of assumptions increase, the uncertainty associa- 
ted with the conculsions reached in the study increases proportionately. If 
the objectives are limited in scope, good alternative systems or other con- 
siderations may be eliminated from the analysis which could result in a less 
than optimum system selection. Sometimes the objectives are stated in terms 
of a specific system rather than a functional need. This undoubtedly eliminates 
worthwhile alternatives from consideration and biases the outcome of the study. 

In addition to establishing reasonably complete boundaries for the system, 
it is necessary to select ■ specific factors to be considered in the analysis. 
Again, this task cannot be defined for the general systems analysis or cost- 
effectiveness problem; however, general assumptions can be suggested, and these 
can be formalised when the requirements of a specific analysis are given. Typi- 
cal assumptions include the following: 

’ The system will be in operation for T years. 

’ All external factors for which the gross cost estimate is less than 
X x & of the estimated cost will not be considered. 

‘ Factors with relative-cost estimates between X-,£ and X-j? will be included. 

i r. 

only if sample data and knowledge of the relationship arc available to 
yield an acceptable degree of confidence in results. 

‘ Fa. tors with relative-cost estimates over X.-5 will always be inolud 1. 

r' 

except that those with leverage effects will be excluded If their 
inclusion would entail an additional level cf analytical complexity 
that would threaten the timeliness of the analysis. 

All a. ’.sumptions should be explicitly stated and .justified by faetua . evi- 
dence. If none exists, the reason for making the assumption (e.r.., mat heirat leal 
convenience, general consensus) should be stated t.o indicate how much audit ional 
study is required and to pinpoint, areas where errors might b‘” introduced. 
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The constraints placed on the system are essentially a set of boundaries 
for the various factors within which the solution to the problem must be found. 
They may include a fixed budget, a period of time, a desired effectiveness, or 
a method of operating with resources. In other words, the constraints asso- 
ciated with the system serve to designate the amount of freedom allowed in 
manipulating system variables to obtain a solution to the problem. 

Constraints can often be used as the sole basis for distinguishing between 
the feasible and unfeasible alternatives. However, a constraint can be so 
stringent that no feasible alternative can be found, and thus there is no 
solution to the problem. There are problems, however, for which a solution 
must be obtained. An example is a person who must have a personal automobile: 

His constraints are that the cost must not exceed $2,000; that the car must be ?. 
new; and that it must have an air conditioner, radio, automatic transmission, 
and a leather interior. There is no solution to this problem within these 
constraints. If the person must have a car, he must relax one or more of 
the constraints until at least one feasible alternative can be found. 

Thus constraints can be used to reduce the scope of the problem, but they 
also must be flexible so that they do not preclude solution of the problem. 

2.3 DEVELOP MISSION PROFILES 

After the requirements and objectives have been formalized, the next step 
is for the analyst to develop the mission description. These mission descrip- 
tions or profiles translate the requirements and objectives of the system into 
specific statements of performance. They in fact describe the tasks to be 
performed by the system in order to meet the stated objectives. Also included 
in the mission profile are such considerations as threat, environment, and 
tactics . 

If the system being studied were a transport helicopter, the set of mission 
profiles would represent all of the tasks that the transport helicopter was 
expected to accomplish. Some of the required missions might be of greater 
impoi ance than others in the set of mission profiles; however, this possibil- 
ity is taken into account in the presentation of results. 

The generation of mission profiles can be characterized by the flow chart 
shown In Figure 2-2. I: ' tially, the operational requirements and objectives 
are determined by considering such factors as the number of missions required, 
the functional concepts to be employed, the enemy threat, the spectrum cf 
environments that may be encountered, and any other requirement as stated In 
the QMR. These requirements and objectives are then translated into specific 
statements of performance -- for example, the number and type of communications 
equipments required for a mission, the information rate and reliability require- 
ments of the communications equipments, and the range and weight constraints. 


2-7 
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MISSION-PROFILE OEVELOPMEMT 


There are several Important factors that must be considered In developing 
the mission profile: 

(1) The mission parameters (or set of boundaries of the mission) should 
be expressed as minimum goals and maximum values. Care must be exercised to 
avoid undue rigidity in mission parameters and each mission parameter should 
be assessed as to its criticality in terms of meet'"g the requirements and 
objectives . 

(2) Postulation of missions is not solely a job for analysts, rather it 
should be dealt with on a team approach with talent drawn from both the tech- 
nical and military communities. 

(3) As the task progresses it may be necessary to eliminate certain mis- 
sions because the risks associated with meeting the objectives may be too 
high. 
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(k) The missions must he sufficient 1 ? detailed to enable the analyst 
test all the significant system parameters. 

(5) The missions must he able to be modeled -- (war gaming/scenarios). 

The final step in generating the mission profiles is to combine all the 
various Inputs into a scenario format that represents the combination of sys- 
tem requirements, threat, environment, and tactics. The missions thus serve 
as transfer functions which relate the sys^m objective to the performance of 
the alternative systems. 

2.4 OITftIR CRITICAL PEIFIIMUCE PMAWETEIS 

There is no precise procedure for defining a system and its boundaries. 
Consequently, all possible variables within the system, their expected inter- 
actions, and their relevance to' the problem being addressed must be examined 
thoroughly. 

If many variables are considered In the study, a valid sensitivity analysis 
can be made. This analysis ’will show which variables are critical (and may 
require more detailed study) and which variables can be ignored. 

Not all performance parameters of a system will be needed in the evaluation 
of a system's effectiveness, because some performance requirements interface 
more closely with the system objectives than do others. The performance re- 
quirements of the AN/APN-70A (a navigation receiving set) are a good example. 

The general description of the AN/APN-7CA is that it is a receiving set designed 
to furnish navigation information to aircraft up to distances of 1500 nautical 
miles from special ground transmitters.* 

Some of the performance requirements listed in the specification for the 
AN/AFN-70A, such as sensitivity, selectivity, accuracy, anc stability, are 
directly associated with the objective of tne receiver. Others, cue!: so seif- 
checking function an j oscillator radiation, raav net b* sensitive to the objec- 
tives and would, therefore, be cf secondary importance in the effectiveness 
model. However, without knowing the objectives cf a eyr*em, it would be diffi- 
cult to distinguish precisely which performance carame* ers were needed t.r 
achieve the objective an'-! which were not. I: one cf * ho requirements of the 
APB-70 is to receive signals wl’hout being ,ed, th»r. an item suer, as 

o.'illator radiation may be an important parameter. Thus it would appear that 
that to talk intelligent .;: abou* * he importance cf syst em-"erfor, nance parame- 
ters, it is first necessary * .•> understand which of the many r-.r&meterc of a 
system are essential "'or the objectives of the sy.-‘ cm.. F.xamr.'.es of several 
performance parameters that may be asseoia*. • .! with different types of equip- 

•MOjrjf n »■ fs f ol 1 t 


* From MIL-P-726B 
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Coeaunlcatloni • riraa. noil* characteristics, receiver lenaitivitiee, 
transmitter power outputs, Input power requirements. Jamming vulnerability, 
and number of channels. 

Computers - accuracy, computational spaad, input -output formate, memory 
capacity, retreval speed, language capability, and analogue to digital capa- 
bility. 

Antenna - gain, coverage, transmit and receiver losees, physical limita- 
tions. 

Another factor that mist be considered in defining the critical performance 
parameters is the concept of controlled and uncontrolled variables. 

Some variables can be controlled early in the system's life. For example, 
the time needed io repair a failed item during the operational phase can be 
controlled to some extent during the planning and design phases. Later, during 
system operation, the times to repair are distributed according to some observed 
function. The expense of changing these repair times Is generally higher in 
the operational phase than in earlier phases. Other variables must be assumed 
to be distributed according to an assigned or predicted function during the 
early phases of life. Later, during system operation, these variables can be 
controlled. Two examples of variables in this category are the mission times 
and mission frequencies for a new helicopter. Still other variables are never 
subject to control — for example, the weather; and there are some variables 
that can always be controlled — for example, pay scales. 

Some variables that cannot be controlled can be influenced through vari- 
ables that can be controlled. A common example is the set of variables repre- 
senting the opposition's reaction to changes in strategy directed at them. 
Another example is a battlefield war game in which player A's strategy Includes 
Influencing player B's moves through the control A has over his own forces. 

The cost parameters will depend on how costs have been defined In the cost- 
effectiveness problem. They may be time, money, lives, distance, or area. 

The criticality of cost parameters is always subject to change. For example, 
fuel consumption may be hardly considered until a certain turn of events limits 
the quantity available. Hence, a miiita'W mirrion -ar. tc conceived of in which 
the cost of fuel exceeds the cost of ammunition even though the dollar costs 
of the two are the same. 

M SYNTHESIZE ALTERNATIVE SYSTEMS 

2-5-1 The Total System 

The term "System" appears to defy unique definition; i.e., one man's sys- 
tem may be another man's component. For example, 8 co«pn-y may have a contract 
to develop a radar set. Within the company this radar may be thought c-f as a 
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system. To tfts customer, however, thie ease re<Sar may be visualised as only 
one element of a larger "syotem" which he Is planning for the control of air- 
craft over a geographic area. This latter man's system, in turn, might be one 
part of a much larger system which has a mission of managing a large mass air 
transportation system. There la really nothing wrong with any one of these per 
sons feeling his own scope of design is a system. However, a definite danger 
is involved in not recognising adequately at any giver, level of consideration 
the relationship of the given system to all potential supersy6tems and the 
Implied interfaces. 

It will suffice for the purpose of this Guidebook to define the total sys- 
tem as it is shown in Figure 2-3. At the center of the total system is what 
is termed the Object System. The Object System may be defined fragmentaily to 
coincide with the scope of development responsibility which a prime contractor 
would normally be charged with. Development of the Object System must be accom 
pushed In consonance with the requirements, constraints, and Interface charac- 
teristics of the world In which it will ultimately operate. This world Is 
categorized into four blocks, or systems, shown in the figure. 



FIGURE 2-3 
THE TOTAL SYSTEM 


A supporting system is one which is necessary to the performance of the 
Object System but not J.n the direct functional line, for example, a maintenance 
system. The environment which would tend to degrade system performance (such 
a. the physical environment, electromagnetic environment, and others). The 
demand ayot.em characterizes the need for which the Object System is being 
developed. In a military situation, *he demand may take the form of a threat - 
more specifically, It might be the approach or an enemy aircraft. For an 
Object System which provides service (such as a communications system) the 
demand system characterizes the potential users of the comnunlcnt, l ons service 
and their habits and technical requirements. Related systems are those with 
which the Object System must cooperate In performing its Intended function. 
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Thus, wo aeo that during the development phase, system analyala should not 
he confined only to the Object System but should be used to characterize the 
tour surrounding system categories. 

System Configuration Synthesis 

After the requirements and objectives have been defined, the critical 
performance parameters obtained, and the mission profile structured, a system 
or set of alternative systems can be synthesized. System synthesis requires 
several ingredients, among which the more Important are: 

, * An adequate understanding of the mission profile; 

■ Ar. appreciation of the ultimate user's capabilities and limitations; 

‘ Intuitive Judgment concerning the effect of combinations of equipments 
operating as a system; 

' A working knowledge of the state-of-the-art capabilities and limitations 
In technical areas from which the potential new system may be drawn; and 

* Common sense to seek the advice of experts, in questionable or critical 

technical areas. 

s. '.;?. l Ba sic Functional Subsystems 

The first step past a mission profile definition is the identification of 
basic functions necessary to any system capable of accomplishing the mission. 
This Identification should be carried down to the lowest mission-oriented level 
For ''xonrrle, the functions necessary to support a long-range search radar are 
(1) power supply, (2) antenna, (3) antenna drive, (^) antenna feeds and match- 
ing, (5) signal formation and amplification, (6) single reception and analysis 
elements, (7) transmit/receive dlflexing, (8) signal processing, and (9) sig- 
nal 11. play. Each of the above functions has a rather direct relationship to 
the mission of long-range radar search. Figure 2 - 1 * illustrates a similar break 
i own for a ground-to-air missile system. This system Is more complex than the 
radar example and In turn the breakdown is not carried to as low a level of 
lefal 1 . The breakdown in Figure ?-•* is shown in block diagram form with the 
••or.riecting lines Indicating only very basic relationships between blocks. One 
mould be careful in taking this first step, not to unintentionally preclude 
any particular technical approach to conf lg.urlng a system. 

. ", . ,> Identify Basic Alternatives 

Once the basic functional elements nupnnrt i ng « ml** ion have been defined, 
•r.ere t s physical realisation that each function car, be enumerated. Table 2-1 
.-•ho**;: nuch an enumeration for the Zenith missile system. The left-hand column 
f the Table is an Itemization of the basic functions supporting the mission. 
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These 1 ar« arranged from top to bottom roughly In a reversed order starting from 
missile detonation. The first of the basic categories contains the configura- 
tion options which could conceivably accomplish the function. Next, Is a column 
for entering design factors which could Influence the selection from among the 
options. An example of a design factor is warhead weight. To accomplish an 
equivalent effect, a non-nuclear warhead would involve several times the weight 
of the nuclear warhead. The next column includes factors relating to the use 
of the system (such things as side effects of the selected options). An 
example of a related mission factor can again be given regarding the cnoice 
of w&rheads. If the missile is to be detonated over or near friendly forces, 
a nuclear warhead has certain disadvantages over non-nuclear warheads. Some 
of the factors entered in these latter two columns will undoubtedly be the 
basis for eliminating many of the possible configuration options. Others will 
simply require consideration in making a choice. In studying Table 2-1, certain 
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comblnatlonu of options will turn out to be Impractical from the interface 
standpoint. For example, the use of thrust vector control alone may be ruled 
out if the terminal flight of the missile is to be unpowered. 

y.5.2.3 Identi f y Useful System Levei Performance Parameters 

The next step in the process of system synthesis le to define a hierarchy 
of parameters which bridge the chasms between mission profiles and functional 
subsystem performance. Figure 2-5 depicts a hierarchy corresponding to the 
Zenith system. At the top of the figure are two basic mission performance 
parameters: kill probability (P„) and time to react and effect a kill. The 
hierarchy in Figure 2-5 conceivably :culd be continued down to the level of 
module and even piece-part output parameters . I 13 not necessary to ao so at 
this stage of development, r.owever, since the lowest leveie in the figure repre 
sent parameters wnlch easily relate, In most cases, to- abatements about state- 
of-the-art and gross characteristics of design options. The parameters whlcn 
are underscored with heavy llnet represent what*, for practical purposes, could 
be called system- level performance parameters. Avoiding the question of what i 
system level and what Is not, suffice St to soy that the underscored parameters 
are called system level parameters because they are the link between mission 
profile statements and -the basic functional elements as defined in Table 2-1 . 





FIGURE 2-5 

HIERARCHY OF PIRIMETEtS 




AMCP 706-191 

? • 5 • j State «of-tha-Art Analysis 

In a state-of-the-art analysis, each critical design area, or component, 
should be viewed from three general points of view. They are (1) the technical 
suitability of a proposed technical approach, (2) time phase consideration 
describing all of the particular tochnlcol approach relating to the mainstream 
development time frame, end (3) the economic Implications of Incorporating the 
technical approach Into the system. Table 2-2 la illustrative of the overall 
Item! which would be subject of scrutiny in determining the state-of-the-art 
feasibility or a given critical design ar^a. 



TAIL! 2-2 


Continuous 
Parameter 
Dev 1c or. 


Discrete 
Parameter 
Dev Ices 


CHARACTERISTICS PERTINENT TO A STATE Of -THE- ART 
ANALYSIS (ELECTRONIC) 


Dynamic range 
Noise threshold 
3aturatio.t limit 
L 1 near 1 ty 
deniilt Ivlty 
<- Frequency response 
Gain 

r 

Isolation 
Stability 
Eff lc ioney 
. . Hysteresis 

J Pulse rise time and fall time 
c | Overshoot 
Discrimination 

Pulse response time i 

Cycle time 

Jitter 

Sensitivity to physical environment 

Sehccic and vibration 
T'es^erature 

Humility, water, lee, snow, rain, fungus, dust 

Radio active bombardment 

Sensitivity to elec * ronagr/'t! • er.vl ronnci.t 


> 


> 


Kune ticna'J 
Character 1 ot les 


Re idabd 1 1 1 y 
Chat a’t .or 5 ,,t 5 -r. 


RFI 

Conducted ♦. rans ! < nts 
Inductive coupling 
Capac 1 1 1 '!• r cup 1 1 tig 

Failure modes 


Development costs 
Evaluation costs 
Production cost:'. 

•Special tools, r re- es. r.a* eyt al- 

tr.stai latlon 
Malntenar/'** rests 
Support c-'sts 


L, Econo r,*: 

I' Character!!:* !c» 
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The major competing alternatives uuai be identified or cynthenized . The 
zariaLies to be analyzed for quantification of these alternatives are also 
identified. Usually, certain alternatives Are well defined at the beginning of 
analysis. One of the major benefits of an analysis is that it can generate new 
1 1 * •*rnnt I ve -- feme of which may be combinations or modifications of existing 
ones — as the analysis progresses. Initial identification and selection of 
/(triable-, generally require screening, e.g., through preliminary sensitivity 
ana is. Decisions on alternative and variable selection should be continually 
r*.— ••xrur.lned as analysis progresses. 

The evolution of a system Is u complex process. It begins with an Idea and 
pro.: cc is through various stages, characterized by Increases in Knowledge about 
the* system until It reaches a stage In which the sy-tem is deployed. With each 
Inc reuse In knowledge "bout the system, some decision is made that furthers the 
sysi-m's definition and reduces the degree of freedom available for subsequent 

i 

io cl:. Lons. At each stage tnore are al ternativeo that can have a significant 
o: feet on the future of the system. Selection of one of these alternatives 
it. a i-u focusing of attention to a new set of alternative**. 


1 

Different alternatives to a system are applicable at various levels of 
system detail. At one level, when action Is necessary in response to a threat, 
the alternatives may include the procurement of a new missile, artillery, or 
air-raft. These are evaluated It: terms of defense objectives and resource 
requirements. Once a decision 1.: made on or.-- of inure alternatives, say the 
missile, attention is focvi.vu or: aitorna' Ivor within -he missile system. These 
arc evaluated In terms of the object iy«r, and resources applicable to the missile 
Selection of this alternative further defines the system's subsystems, and * 
attention in' then focused cn alternative.; within the subsystem, say the eoaauni- 
:< tion em. This process continues until the system is completely defined... 

2.3 DEVELOP HARDWARE CHARACTERISTICS 


The hardware '.’haraclerls.il--.. frr onrh t.ten.-c ! vc system are determined by 
neces ■; l.’.y for me‘-t 1 r. - i-.p*--*lfi • r- jul> e-nen* ;; witr.ln the mission profiles.. 

Ti har lwar * :hn>'-, • t.- • r 1 ..M may be sul • n ln‘ re-system trade-offs to 

•o’ Imt.-e each a 1 t-rn.i t ! . (T'’- 1 'le-itff s. V- dinousaod ! r. spot ion 2.12^ For 

• -xamtj .<•, * in: of n h.ri i copter « ' r? rniv - au be determined by the* requirement 

to carry n glv -i number of t r> •*-.ps. . The ho;. * power for the engines can now 1 e 
!*•' rr.tned v •/ * one- J 5«?r • r - * he >;• f - h*< airframe. troop'c . > -v. ,, anti the 

! an! hov-r re-pjl r>- ■;* Ir. *!-.<•• r' *r t« t!*- flies. 7n *urn, this i or sept w-r 


■ i 7 


ip • '* engines, and 


• r-r !■ 


P . -.-•’ ■ 


t**s ir-it le 
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(Essentially, a single -engine system coats less in maintenance arid initial 
.Investment, while the multiple-engine is safer and lees venerable to enemy 
action.) 

The hardware characteristics for a communication system would be deters ined 
cy the performance requirements (range, channel capacity, system reliability, 
etc.) and by the constraints on weight, mobility, etc. 

M ISTAILISN IASIS FOR EVALUATING EFFECTIVENESS 

Th-- ultimate output of any syatem Is the performance of some Intended 
function. This function is frequently culled the mission in the curse of a 
weapons system. For other types of systems, it may be described by some system- 
output. characteristic, . each as satisfactory .message transmission in a communica- 
tion system or weather identification in an airborne weather radar. The term 
oft-.-n used to describe the overall capability of a system to accomplish its 
mission Is system effectiveness . A more precise definition of this expression 
w 1 ! be .-.iven later, bur. for the present It is sufficient to observe that system 
< l veness is related to that property of system output which was the basic 

reason for buying the system: the performance of some intended function. If the 
ayston is effective. It performs this function well. If it. is not effective, 
attention must be directed to the system attributes that are deficient. 

Because of the variety of systems to which it Is applied, system effecti Va- 
nes;- ha.: been defined in a number of ways. The WSEIAC* effect produced the 
fo . lowing general definition: 

Sy 'em Effectiveness Is a measure of the extent to which- « ryst.om may 

b- expected to achieve a set of specific mission -equ: remente . It Is 

•* function of the system's availability, uepenciubi 1 : > y and capability. 

The basic approach for evaluating the effect 'venose c! a system can be 
empirical or analytic. * 

Tie- emptrica ! approach consists, of r.o I 1 e--; ». 1 nr. data ana evaluating system 
if ff l"/ cnees by observing performano -haract <-rlst of systems- It, ,t.h«- 1'lc.d. 

Howe'.vr, this approach is applicable only - for s/.- 1 »*ms that are in. as, advanc>--.i 
f their life eye lea,. 

Th • analytic approach, however, deer. r.« t require t ha* the syr, : cm be l v . 

■>>’ is * *-r, , .. The approach la baaed on the -o r.; * ruct ion of a mat.hema' i -a i model 
t.r-a* in ludes predict tons of system ehnra.o or! s • I within t.h» .-.'.n:-* rt« Snts 
l rf- ■ ::<• i by the arniy ' . 

i • >• >i-< rj ."»y..:t.« m peri ormntvo Manua : ** s.v; • • nls >.b< >;• 'Vo- «nt» »y< j appro, a : 


•WroK/a* Vloap'r. ” f v •- ter K! ! oi t i v.-n- "... ;r, J> . 

* •iiAVMAT }’ : • > . 


A v 
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"Purely empirical or purely analytic methodologies are, of course, 
not very useful. The former yields highly authoritative data too 
late to be useful, while the latter yields answers unsupported by 
facts. In practice, a balance is sought. This balance will normally 
change during the life cycle of a system. As data about the behavior 
of the system becomes more available, the analytic model graaually 
merges into an empirical model; as the data becomes more available 
and as confidence In their value increases, statistical sample data 
supplant the assumptions." 

The need for analytic models to predict system effectiveness is based on 
the '"-ec! to evaluate the effectiveness of a system before it has been in the 
field for many years. Although the empirical methods are required to provide 
inputs to the systems analysis/cost -effectiveness process, it is the analytic 
approach that is the most important and useful. 

One analytic effectiveness model thet has generally been accepted is the 
WSEIAC model. The WSEIAC model is based on a breakdown of the effectiveness 
parameter into three major components — availability, dependability, ana 
capability. Availability is a measure of the system condition at the beginning 
of a mission; dependability is a measure of the system condition during the 
mission, given the system condition at the start of the mission; ana capability 
is the ability of the system to perform its mission, given the system condition 
during the mission. 

To apply the model, given a mission definition and system description, it 
is necessary to delineate .possible mission outcomes and significant system 
states. The availability and dependability measures are then related to the 
possible system states, and the capability measure relates these possible 
system states to the possible mission outcomes. 

For the very simplest, of cases, in which a system must be in either a 
working or a failed state, the measures of availability, dependability, and 
capability represent the following fundamental quest if ns: 

" Is the system working at the start, of the mission? 

‘ If it is working at the start, wi.il It '••ntlnue working through© 1 ., 
the mission? 

' If the system works throughout, the mission, will it actual ly achieve 
mission success? 

As Uit systems considered become more .employ -- e.g . , wh«ni there are 
more tnan two possible system" states, and surf, factors a. in-mission repair, 
degraded modes of operation, mult i-m! is ion requirements, enemy "sunt ermeasures , 
and natural environment are to to quant if! eu p 1 emen‘ s in t he mode i -- these 
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questions say be too diffi- .it to answer with simple model construction, but 
they still represent the fundamental VSS1AC approach towards evaluating effec 
tiveness on amission-oriented basis. 

The general framework for the analysis of system effectiveness is given 
in Figure 2-6. The elements of system effectiveness are discussed briefly in 
Sections 2.7.1 through 2.7.3, and are outlined in Figure 2-6 a. 



FIGURE 2-1 

SYSTEM EFFECTIVENESS FRAMEWORK 



FI6URE 2 -II 

ELEMENTS OF EFFECTIVENESS 










1 '$ 


2. 7-1 Availability AftICJ* 79M91 . v 

The concept of availability concerns the system's condition at the start 
of a mission. The WSEIAC definition is as follows: 

Availability Is a measure of ♦•he system condition at the start 
of a mission. It is a function of the relationships among 
hardware, personnel, a«d procedures. 

Availability has also been expressed as the probability that the system 
Is operating satisfactorily at any time, when used under stated conditions. 

Poo this guide book, the more generel WSEIAC definition is used. 

Application of the availability concept requires cloar definition of what 
Is included In the system and of the system's mission. Availability Is a specific 
measure; It Is therefore usually necessary to define more "han one system and 
Its associated mission and to define availability for each :aee. 

In estimating a system's availabil? ty, care must be taken to consider now 
the "system" has been defined and bounded. For example, with ‘■‘egard to a group 
of 2 e tanks, an analyst may be equally interested in the availability of a 
single *ank, several tanks, and the group of tankc. For the ? c tanks, then, 
seve:sl systems and several corresponding missions can be defined, ar.d the 
meastue of availability will be different In each case. 

Assume that the system is defined as a group of four tanks (frtm the 25) 
and that the mission Is to attack as a group. If at the start, of a s lesion one 
tank is found to be Inoperative, the system will be counted as being in an un- 
available state. However, the group mission Is not necessarily aborted, since 
it mlgt,* be possible to draw a replacement from another group . In this case, 
then, aval, utility is considered zero, ar.d yet it is no: necessary *o abcn* the 
mission. 


On the e*her hand, If the sys‘em consists cf all P ; -auks and ‘he system's 
mission Is *o provide *ar.ks for specific r'sslons as req.'red, availability 
I mlgh* be a measure of *• e r. tmber of * arks uva’lable for assignment at any time, 

f It Is not a measure of ‘ he number of opera - '.oral ‘arks, sl . ee aval laMll’y may 

I be zero -- for example, when °C pereer.* of • v -e tanks are perform! rg specif' 

missions and 20 percent are inoperative. In this case, it la ai.-jr possible to 
conceive of an aval lab 1 11* v of zero when ell 'arks *r * l e group are opera' 1 oral 
; and performing ‘heir assigned missions. 


In ano‘her s'.*ua*!cr, ‘he -ea; re of '.feres* may '•e **.e • .mber of ‘arks 
‘ha* are operational a‘ anv *lme. The sys’em Is •: e er* i re pro p of ‘arks, and 
the system mission Is *o malfalr. all ‘arks In opera* ‘.oral s‘a* .s . In ‘Ms vase, 
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tt is possible that availability will ba IOC percent and yet a specific mission 
will be aborted because all tanks are already performing specific mission^. 

2 - V . 2 Dependability 

The second major element in the system-effectiveness framework is depend- 
ability. The W3EIAC definition is as follows: 

Dependability is a measure of the system condition at one or 
more points during the mission, given the system condition at 
the start of the mission. 

The nature of this concept is similar to what is commonly referred to as 
reliability, except that reliability is usually defined as tnt probability 
tnat a system will perform satisfactorily l'or at leaBt a given period of time 
when ,sed under stated conditions. The more general WSF.IAC definition is used 
herein since the system's maintainability characteristics also influence its 
dependability . 

Ac yt* th the concept of availability, appli -at. ion of tne dependability 
concept lequlrea an exact statement of trie system's composition and mission. 

A single estimate of dependability for* the system, in question may not 
convey all that should be known about it and related systems . For instance, 
when tt la said that the probability of t" ■ in; a target with a weapon from t»r 
attack helicopter is 0.90, It is impllt . • rat a favorable reoul - will be obtain 
ed, with probability O.OO. This statement also indies’ es ♦. na • there is a V. 10 
probability that the target will r.o' be r. i • , but does not g! at. y information 
abou‘ the causes or consequences of s fa'l n-, bid • ».« f-»»l'.or>ni er crash 
before reaching the target? Did 'he weapon o' rav in’c a fr’*r dly area" Did 
the weapon fall 'o release from 'be he i: cop' **r' War 'be • el : < op’ er gr*o .Tided 
because of bad weather? To evaluate 'he nvs'em properly, i * in necessary to 
consider the circumstances surrounding a fail. re. 

When the probabl 1 is' i c definition of reliability firp' care •r.'o genera', 
use several years ago, one of the mos* impor’an' prerequisites 'o applying i t 
was ‘•hat "failure" be explicitly defined. This is impl'-i ' in • ‘ e definition, 
since to say what is satisfactory performance under giver cor.d' ‘ ‘ or.s , i* ‘is 
necessary to state carefully what la no' sa* lefoe'erv perforrgm'-e (i.e., fa fur 
under 'hoee same conditions. 

The problems become more complex in ' ’.e -as* f m .1' '.pie systems at a 
mise’or.s. Interest may be centered on ‘r.* depet debt 1 i ♦ y of a single commu- 
nications system, art equipment In the system, or a gro up of o;,s'*ms . • Each 
sys’em may be capable of a number of d! fferer* missions, r.~-r.*- nor* -r;*‘. a! 
than others. Combining the dependab i 1 ' y f ' g ;r*r for ail sys* n»-.d m'rssior.r 
tends to obscure 'he favorable or t. favorable •• ar.* - eri n' ; t <■:' a,-, irdivldua: 
system or mission. On the ether hand, if attention l.i focused on a single 
system or mission, the tendency is to ignore the favorable or unfavorable 
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characteristic* of ths related systers and mission. Measurement of depend- 
ability requires Identification not only of each critical system and mission 
but also of the Implications of success or failure in each instance. Then 
the several estimates of dependability are displayed in the form of a vector. 
Although this type of display does noc facilitate the decision-mgking process 
(if anything, it makes the proceaa more difficult), it does lead to more ac- 
curate dec la Ion -making. / 

2.7.3 Capability . * 

The third major element in the system-effectiveness concept , capabl ill ty , 

Is defined In the WSEIAG report as follows: 

Capability la a measure of tr.e system’s still ty to achieve 
the mission objectives, given the system condition during the 
mission. Capability specifically accounts for the performance 
spectrum of the system. 

A similar concept that expresses ’.tie characteristic of a system Sr. 
probabilistic terms is Design Adequacy, which has been defined as the probability 
that a aystem will successfully accomplish Its mission, given that tne system 
is operating within design spec 1 ficat ions . In thie guide book, the more general 
WSEIAC defini - 'cn is used, br* in specific instances oystem capability may be 
expressed as a probabll' ■ It should be noted that capability or design adequacy 
is not solely an inherent characterin' 1 of system hardware. Capability depends 
to a significant degree on 'he mission assigned to the system. A system that 
waa designed *o accomplish a specific ’ask may very well have a high capability 
for that task. (In probabl Hat * • -errsi. pem.aps ’he Design Adequacy is unity.) 
However, If it is used for a tr.c:«* difficult or complex mission, its capability 
for this new mission may be low. 

The measurement and predict*. or of nvn'err. capaMll*y is a rather complex 
problem *n Itself. The 'ffic.ltiea * r* rod -I by muV‘ modal systems ar;d 
multiple missions in -he appl * -a* i or, of avo i '.til • ,v and dependability ocncep* * 
are also present, ip the use < * * he ‘-apat.i 1*. • y eortcep* . Further, 'be capabl 11 ty 
concep' has not yet been developed *o • he ex* e* - ♦ hat ;■ e ar be quantified by- 
standard techniques. 

The system analyst mus- be par* ioularly careful * o diet inguish clearly 
be* ween capability and dependab i 1 i * y . I* .was r.er.t *, oned earlier -hat failure 
and the clrc ums-ances surro erdi-g fall r« ".s' is exp] * c 1 1 1 y defined before 
*-e dependab ill* v cor cep* ’at be app . * ed . The sar.e > • rue of the capa! i 1 • y 
. "onrider, for example, a 1 -• or -• r - k ' ' rt 1 low-cc* *ha\ o r ' r rs a* 
no mpt, on a 4 o* da v (lio.op), .pen impa ■■* of ' • e • • re w‘. ' 1 a iag,gr.d id e it 
* *.e pavemen* . Is the failure %• * rii .• ed • r a la'k of dcpet.dat i 11 *y or cap- 
abili*/' If the 'ire f system) wag designed for hi? 1 -speed and :• i gt. - • empe ra ’ re 
environments, and to withstand high impact leads, then the blowout (failure) is 
attributed to lack of dependability 'reliability) since the conditions rf 
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satisfactory operation ware defined to Include that* mtrt environment*. On 
the other hand, i i the blowout occurs on a tire designed to operate In such less 
severe environments (perhaps 40 mph at 80 °?), then the failure is attributed 
to lack of capability.* In the first case the tire (systea) had adequate cap* 
ability, but Its depandabllity was low. In the second case the systea* s 
dependability say have bean high, but its capability (for the particular ale* 
sion) was less than adequate. In either case, it is important to note that the 
system effectiveness of the tire la below the acceptable level. 

2.1 KNN KUMfS #f CF'ICTIUMSS 

The measures of effectiveness** for * me systems are easy to obtain and 
commonly accepted — such as ton-miles/ -ay for a transportation system, and 
single-shot kill probability for an anti-tank missile. However, for other 
systems, including many electronics systems, ^ nc overall measures of effective- 
ness have been develo'ui. For example, the effectiveness of a communication 
system may be measured In terms of Information rate, information reliability, 
system reliability, and system availability however, it Is not possible to 
combine these four factors Into one overall measure of effectiveness. This Is 
not necessarily a disadvantage, oeeause the decision-maker can still make- a 
choice even If the effectiveness la presented as four separate numbers; of 
course, the choice may not be as simple or clear-cut. 

The measures of effectiveness are subject tc change with time In a battle- 
field situation. At one time, effectiveness may be measured in terms of the 
damage inflicted on enemy supply routes. At another time, It may be measured 
In terms of how long It takes tc Intercept an intruding tank column. 

If the mission profiles hv/c been specifically defined, the problem of 
defining measures of effectiveness it greatly reduced In that the effectiveness 
can now be expressed as a probability e; accomplishing ail or a given part of 
the system mission, in other cases, where a set of specific mission profiles 
cannot be obtained, the effectiveness must be re.ated to the physical character- 
istics of the system -- for example, rarge, channel capacity, speed, etc. 


In the operational situation, a’-trihu* i;ig unsatisfactory performance to a 
lack of either 'ap'iM’l*y cr leper.uah : iity Is cfen based on the frequency 
of unsatisfactory performance. Frequent failure in norma 5 operation makes 
capability suspect, while infrequent failure makes dependability suspect. 


•• Also -'ailed criteria for effect 1 

t Much work Is currently under v«, 
t lv»ness . 


measures of *ffec- 


are.a to define 
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2 . 3.1 general 

A system model Is essentially a mathematical, logical, or phy-leal repre- 
sentation o t the interdependencies batman the objectives and the raaourcaa 
associated with tha system and its use. for dealing with the effectiveness of 
complex systems, the model is usually in me fora of mathematical equations 
(mathematical model) or computer programs for simulating system operation 
(simulation model), or both. 

On the assumption that a set of system objectives has been translated into 
an' optimization criterion, the model builder is required, minimally, to con- 
struct a model that will enable quantification of the critical effectiveness 
and cost parameters as a function of the resource variables. 

The overall cost-effectiveness model Is usually one that consists of several 
sub-models, each of which may be based on models at still lower levels. Figure 
2-7 Indicates one means for sub-model classification. It should be noted that 
there are many other schemes for classifying models. 

There is, naturally, a great deal of interaction amonst the eub-models, and 
model Integration Is required In the same sense that system Integration Is 
required . 

Constructing suo-modele (and integrating them into an overall model) Is, 
for moat "real world* situations, still more cf an art than a science, largely 
because the validity of the model cannot be tested tnreugh controlled experi- 
mentation; thus the collaboration of people with wide experience In the areas 
of concern is an Important requirement. 
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2.9.2 Assumptions 

All assumptive* required ror th« model should be explicitly ststed end, If 
possible, supported by factual sviuence. If no such evidence exists, it is 
advisable to state the reason for the assumption, e.g., mathematical simplicity 
or consensus Of opinion, in order to indicate the degree to which the assump- 
tions will require further Justification and to pinpoint the areas in which 
errors might be introduced. 

2.9.3 Adequacy 1 

A model must bo adequate in the sense that all major variables to which 
the solution is sensitive are quantitatively considered whe:e possible. Many 
cf these variables will have been preselected. Through n.anipule** ~r: of the 
model, some of the variables may be excluded or restricted, and others may be 
introduced. Non-quantiflable variables must be accounted for by modification 
of the solution rather than by direct Incorporation into the model. In this 
sense they are quantifiable. 

2 . V . Represent at lver.es s 

Although no model can completely cup. te the "real world", it is required 
that the model reasonably represent the true situation. For complex problems, 
this may be possible only for sub-parts of the problem, which must be pieced 
together through appropriate modeling techniques. As ar. example, analytic 
representation may be possible for various-, phases cl' a complex maintenance 
activity. The outputs i'r.-w these ana-1 y: (•:. may then be used as Inputs to a 
simulation procedure for modeling the <.oir.pl «•’.«* maintenance process . 


? . > . ‘ . Uncertainty 

The various types of uncertain’ ; ir.v; *,r. the problem cannot be ig- 

nored, nor can they be "assumed" out; • i ■.»>;,• r.us’ be faced squarely. There may 
be technological uncertainties involved wit*, some of the system alternatives, 
operational uncertainties involved wl’.*. planning and • arrylr.g out the mission, 
uncertainties about enemy strategy and !/’. 1-r,. and s ’. *' ‘ !■ • leal uncertainties 
governed by the laws of chance. The simplest «| rr r « !. is to matte "best guesses", 
but this may lead to disastrous results, sir.-* the probability of guessing 
correctly for every uncertainty is quite small. For carer, involving statistical 
uncertainty, f unc fcions -of - randem-var lab 1 *c • r.eory or su.h procedures as Monte 
Carlo techniques may be used. For ’he other type i cf un or’ aintier,,* the general 
approach la to examine .il l major cor.* ingcr.c tee ar, 
of f«* • l veneon p-> raise’, ers. The op 
for *. •• ir. tne evaluation of the re* 
opr.er.t js o r do- is ion theory and gar.* 

’ion of a ie union mcd«l Sr. *n«r,« -a 
bos’ for different c-.:r,* luge- 


i r.g cr.c tec a r, ’ c cap .to 

result ant 

C 08 t - 

r a * S r, . ri’erl r, , ’her, 

, , must be 

adapt able 

< f ; rjs* »fr * * ..,. ri o: ; . 

■re n u 1 1 r. . - 

The devel- 

her ry t e.'i rr se?t appi 
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2.9.6 Data 

The availability of relevant data piay j an important role in the develop- 
ment and application of a model. Data are required to support assumptions, 
select alternatives, and define constraints, as well as to define the oust arid 
effectiveness constants In tne proposed model, since missing cats may prevent 
valid model application, tne model builder should investigate this possibility 
early In the model development stages plan to ottair. missing data or ad- 
just the model accordingly. If a great expenditure, of time ana money if re- 
quired to obtain the nectssaiy data, the analyst may te forced to weign tne 
risks of using what Is available (and making sjeotesary *i.s«mp'J ont> ) agairtt the 
value received In *•**»*■«•« foe ♦ >.,» -wi# of the ae.ra-oollr.~ti ••*> ana analyst* 
effort. 

2.9./ Validity 

The final test of the mode* is wnein« r <r n> • it yi>- fh< nest ».yt : «•»:.. 
Unfortunately, this determine.? i« n can t •• mad* cn.y after i.yrusp.r are *tj «-i 
and in use, if it can be made at all. j><.w»?ver, • rtain qua.* r i one will dit l- t* 

weaknesses that can be corrected; 


(1) 
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- are rwcultr. 
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;• ‘ i-u* 

a.y.r i'Uirajr.etorf: art 

(2)* 

Sensitivity 

- do 1 
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i‘ . • * ■ i * ■ 

wj • •; ' VT.,. 

:. . t v» output "hang* ;• 
i hi U 
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Plausibility 
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: 'tp/sr ' . ' * ' 

r 4 .‘fi r , 

•? 

'-! •;?;..? i hs . 

; .a'. , e spin * !.< 

(6) 

Sultabil l?.y 

^ „ * r , • r» ■ 

A* * . , ! * i J 4 - 

• *.'• ‘ * ♦ * V. * 

* ; • ' 1 V « ; f. ; 


2 .9.8 Typed of Model -- 

There arc several types ©? ~ ' • . .n .i.v ra * ?.*■ tn‘ ; a. m- delr, gam *, r.g 

models, simulation model s , ar. i r.a . » • • r • T.'co m< -:«-i r *•»•« 

described below, 

2.9.6. 1 Mathematical Model-? 

Mathematical models ere -t<iro •<:'<, ■ •• • :> • * • < ■. • : 

to represent tne chare- 'erir-’i - ‘ , ■■•< r n ._ , «, ■; 

of the four categories. Tt.e ha” I ' ' • ..-»*! • : : a; ‘ t •?• , 

hypothesis to the analysis of ava 1 a; • >» , ? • #,« a. w h» re? ! t 

generally provide a grea? deal d f :• • ' ? ’ ' * ;■ . ? •' ->• . 

simplifying the real-world hhVl r, * ; a .•••>! >• •• tel nr.-; *»<•>• 

decreasing the rep r-- cent at ; ver.»ss 
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2. 9*8, 2 gaming Models 

In the gaming-model technique, operations and resource usage are simulated 
through scale models, computer programs, or physical analogs. Personnel In 
operational decision-making capacities are participants as an integral part of 
the model. Examples of gaming models are military Mar games and the uae of 
aircraft simulators. 

2. 9. 8. 3 Simulation Mode ls 

In the simulation-model technique, all aspects of the system, its resource-.: 
usage, and its operations are simulated in an abstract form, usually through 
computer programs. The basic operational flow is structured and probabilistic 
paths are determined through appropriate random-vaiiaole generating procedures. 
Such computer models are popularly called Monte Carlo procedures. An example 
is the simulation o? component failure and repair times to provide estimates 
of system availability and maintenance and logistic requirements. 

2.9»8.k Operational Exercise 

In the operational exercise, actual system' resources Are Used, generally 
in a simulated operating environment. Examples are a controlled experiment of 
weapon firings involving military personnel and resources, and e military field 
maneuver between red and blue forces. The costs of such exercises are generally 
high, and thus the number and extent of the trials must necessarily be limited. 
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1.(1 flCffim (mtTlHKil CIMIMRS 

The basic WS8IAC sjattm-affaotlvar.aaa aquation la the produet of an 
availability vector, i expandability matrix, and a capability vaetor, vhloh 
are defined as follows: 


A - [a r a 2 , ...aj, the availability vaetor 

a. - probability system Is in state 1 at the beginning 
1 of the mission 


d ll d 12 . . . d ln 
d 21 d 22 ' * - d 2n 


D 


d nl d n2 . . , d nn 


, the dependability matrix 


<5. . - probability of a system-state transition from state 1 to 
J stage J over a fixed period of time 




the capability vector 


i 



o. - the capability of the eystem for performing the mission, 

J given the system is in state J 

A typical term of the product Is A D C 

a l d iJ °S 

where 

Sj is the probability that the system is in state i at the beginning of 
the mission 

4y is the probability that the system will make the transition from state 
1 to state } over a fixed time period 

Oj is the probability (or expeoted value associated with mission 
aooomplishment ) that the system can perform its mission, given state j 

It is emphasised that the VS8XAC Model is not a self -contained, directly 
applicable mathematical equation for effectiveness. As stated many times in 
several ways in the WSHIA' Task Group II report, the "model" is actually a 
framework for effectiveness qv>an»f .o; tJon — a basic routine for constructing 
an appropriate model. Altnouga the mode framework, represented by the 
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product A £ C , can in tome instances bs um 4 directly , this slmpls product 
will not work for « particular system-mission omsblnation. Xt was not intended 
that this produet to always dirootly applioatlo -- only that tho element* of 
availability, dependability, and eapability bo incorporated in each a manner 
that the model framework could be applied. 

As a simple example, t. v .« product A D 0 in actuality is based on the 
assumption that mission performance is evaluated at a single point in tlM — 
the end of the mission. Por many oases, this ie not reasonable. If D (t p , t # ) 
is defined to be the dependability matrix over the time interval (t p , t g ), and 
if the Markov assumption holds, l.e., 

[»<V V]-f*<V *i>] [2<V t.)] 

for all t 1 such that t r <t i <t g , then the effectiveness of the system at time 
is represented by 

B(t k ) ■ A £d (0, t k )][c (t k )] 

Xf the mission is one In which continuous performance is required over the 
mission length t m , the effectiveness of the system, assuming well-behaved func- 
tions, n*y possibly be quantified as the time average of E(t k ) — that lu, 



Note that if at eaoh performance time the capability co-efficient c j equals 
one, end if state J belongs to the set of satisfactory states and is zero other- 
wise, the above equation for B reduces to the expected fraction of the mission 
performance time that the eyatem it in e satisfactory state. 

An extension to the W3SXAC methodology is necessary if the Markov assumption 
does not hold. In this oess, the capability matrix must be written as an !Mf 
Matrix (N - number of syetem state*), with an entry for each atat* transition. 

Exhibit 1 presents e system-effectiveness problem, with its solution, that 
illustrates the application of this technique. 
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EIMMT1 

EXAMPLE OF SYSTEM EFFECTWEIEJS 


Two communications A and B, ars used elsmltane- 

ouely to transmit Information. Should either of the systams 
fall, the rmulnlng one la capable of tranmslttlng alone (A 
and B are statistically Independent). Failures in either {or 
both) systems are not repaired during a traiumlsslon period, 
but are repaired during a period when the equipments are 
normally shut down. 

A transmission will be started whenever at least one of 
the systems Is available (In other words. It is not necessary 
that both A and B be in operable cocditlsn In order to start 
a transmission) , 



The respective mean failure times, assn repair times, and 
bit rates for A and B are given below t 


arete* 

Mean Failure Time, ? 

Naan 

Trsnrmlaalcn Bats, r 

A 

B- 

12 hours (exponential) 
24 hours {exponential) 

8 hour* 
6 hours 

120.000 bits/hour 

100.000 bits/hour 


A normal transmission period consists of 3 uninterrupted 
hours. 


f » 
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What it the effee tivanaaa of A tod B c cabined, tf 
effectiveness l* defined u the probability of tratmlttlng 
at laaat 300,000 bits during a normal tranaBlssion par lad 7 

mwi 

What ia the effectiveness of A and B eaablnad. If 
affaetlveness la daflnad aa tha expected (are rag#) naaber of 
blta transmitted during a normal tranamiaalon period? 

WtlttK) 

What la the answer to dotation 1 if both values of 7 
are Increased 50g? 

Mi»4 

tlhat la the answer to Question 2 if both viluee of f ere ln- 
cfeeeed 50gT ' " 

■mi 

What la tha ana war to (fcieetlon 1 If, inatead of changing 
the valuee of 7, the value* of I are both decreased JOgf 

WtiMi 

Whet la the answer to Question 2 if, instead of changing the 
valuoe of 7, the values of V are both decreased JOgt 


For all queatlona, 
designation will bat 

tha ays tee state 

Configuration Ctate Wunber 

A • B 

1 

A • V 

2 

% • B 

3 

1 • 1 

k 
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EXHIBIT 1 



BOTSi All calcuiatl ons rounded off 
to 2 pieces u they occur. 


i «*• availability (A) of e system 1. the probability 
that * eyeten te operating at any point In tine and ie given by the equation: 


f + * 


In particular, the availability of subsystems a a and Kg are as follows: 


*a ■ irh ' o- 60 


24 


*b " 3TTS " °* 80 


Definition: a^ * p(etate 1 exists at start of transmission — a function 
of the availabilities of subsysteae A and B): 

*1 * < a a> jV* * (O* 60 ) (0.80) - 0.48 

*2 * < A A> ( x *V * < 0,2 °) - 0*12 

aj - (1-A a ) (Aj) - (0.40) (0.30) - 0.32 

•*. - (1-A a ) (1-Ag) - (0.40) (0.20) « 0.08 

where A a and A^ are the availabilities of subsystems A and B, respectively. 

& Matrix: £ - { 0.48 0.12 0.32 0.08 ] 

DeoefUablllty Caloulatloae : The aeaeure of dependability that will be used In 
this example is the reliability aeaeure at socle ted with the operation of subsysteae 
A and B. Reliability, then, la defined as the probability that a system will 
satisfactorily perform its functions for a given period of time. Because elec- 
tronic systems are being considered in this example, the reliability function le 

t 

assumed to be exponential and le given by the equation R(t) • e ^ where t la 
the mission time. Thus the reliabilities for Subsystems A end B ere ee follows: 

t , 

" T - JL 

Reliability^ (3 hours) ■ • A » e ** • e“ °* 2 ® • 0.78 

ReliebUlty B (3 hours) » a 8 » e ™ *- *“ „ 0.88 

• Reference Multiplication law, p. 4-5 of this guidebook. 
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Fleaental Transition 

p(A -A) 3 - 0.78 
p(A -»X) 3 - 0.22 
p(I-a) 3 - 0 


BBBKT 1 

Probabilities i 

p(*-») 3 - 0.68 
p(» -*) 3 - 0.12 
p{l -* ») 3 • 0 


p(I — K) 3 - l p(f — B), - 1 

Definition: • pjfoln* froa at ate t to state J}j 


State Transition Probabilities i 


^ - (0.78) (0.88) 

- 0.69 

Sjjg - (0.78) (0.12) 

- 0.09 

<1 13 - (0.22) (0.88) 

- 0.19 

d l4 - (0.22) (0.12) 

- 0.03 

<12! - (0*78) (0) 

• 0 

<*22 - (0.78) (1) 

- 0.78 

dg 3 - (0.22) (0) 

• 0 

<*24 - (0.22) (1) 

- 0.22 


d 31 - (0) (0.S8) - 0 

djg - (0) (0.12) - 0 

<*33 - (1) (0,88) « 0.68 

<*34 - (1) (0.12) - 0.12 

<*4! - (0) (0) - 0 

d 42 “ (*) * 0 

<*43 - (1) (0) - 0 

044 - (1) (l) - 1 


0 Matrix i 0 - 


0.69 

0.09 

0.19 

0.03' 

0 

0.78 

0 

0.22 

0 

0 

0.88 

0.12 

0 

0 

0 

1 _ 


Capability Calculations 

Definition i C 1J • p [transnlttlnc * 300,000 bits under 
the state transitions (1 — J) 3 ). 


•ll-l 

c 3 i • 0 

®12 " 1 

0 

■ 

ft 

0 

°13 “ 1 

°33 " 1 

0j4 - (see below) 

034-0 

°21 " 0 

« 4 j - 0 

c 22 " 1 

042-0 

®23 *° 

0 

■ 

? 

Oj 4 • (see below) 

°44 - 0 


2>3% 
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* j 9ot and mmUmt notat*oml oolotlom 

• Tra wl— Inn rotooi p a , r # (constant) - Ml— ion tiass T 
. fallen rttoo t lj| (•xponootlol} . Conditional r^T > Pj r^T - p 

. BUa to — tmanlttodi 9 



• 0.19 (for tha nuabsrs Involved) 





• O.59 tf— tha maters involved } 


(mit a«u— Ion e la —I7 oonaittvo to roundoff 
tnont 7 pin— ontrtsa on netted to 
onoun 2 plooo too— 00 * of naolt) 

1 1 O.99" 

i 0 0.19 

0 1 0 

0 5 0^ 
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The equation 1« glvtn by< 


4 4 


1 - I Z*» *M ***““ 

1-1 J-l 


[ V J 1J *11 


Z d 2J c *l 


1-1 x 


* 

Z "*J a S3 


m 

Z 441 •« 


X **14 *14 * (0,69) (1) v (0,09) (1> * (0 * l9> (1> * (0 *° 3) {0r95) * °* W 
J-i 


Z 2*4 **4 - <°> 4 ^• 7S ' tl) ♦ < 0) ,0) ♦<°‘ ast > <••«) • °* to 

4-1 


£ 6+4 «J4 * (0) {0 ) V(o) (0) ♦ (0.18) (? ) + (0*12) (0) - 0.86 


T % °*j • (0) {0) 4 {0) {0> + (0) (0) 4 {1) ' 0) • 0 

>i 


s . fo.48 0.12 C. 32 v.oe 2 X 


*o m 
-°* * 


• 0.18 ♦ 0.10 ♦ 0.28 ♦ 0 • Q.d6 * 
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ft* only ohan*t trm 0a**tlon 1 ooour* in tb* arthodoloajr, definition*, an* 
Mlwlittn* related to tha Cjj Ifctrix. 

teflnltlcm 7} < V 1* tha M* failure tlaa of * arstae 
tntjoct to tha ftcaation that la k ao m to 
has* (UM ty tlaa I. 

fsr an w qponat tl al arataa idtfc fall or* rat* x. 



ft**, tin xadltlsnal naan failure tlaaa for A ant n arai 
7* j < 7 « 1.4 hours 

(for feh* au a ta ra lsvolv*) 

l^J < * » 1.5 h*«r* 

summon f j < f - the o o$ mtmi Hi* *f hit* 

iraauadttel if « ^ * a e aMLa^ la t ow* 
te haw fall* If tlaa *. 


ft folio** ttnotli that 

t| -it* »7J < *) r (5) 

ftaa, for qrati— 4 *4* 

f A ) < a m (180,000) (1.4) - 188,000 toll* 

V,} < 3 • (100,000) (1.5) - 150^000 bate 


Saflalt&ont B| » T - «9Mta4 can to tr of felt* 

uomO^W 1/ a arataa *ims ha* art 
fall* If tlaa f 
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It follows directly that 


so that 


Fj » T ■ rT 

£ a | a 3 - (120,000) (3) - 360,000 bits 
Sgl * 3 - (100,000) (3) - 300,000 bite 


Definition: - expected number of bite 


naxan i 


( 6 ) 


( 


transmitted unaer the transition: 
(!"* J) 3 


u - 360,000 4- 300,000 - 660,000 

c 3l-° 

12 - 360,000 4- 150,000 - 510,000 

C 32 * 0 

13 - 168,000 4- 300,000 - 468,000 

C 33-° 

l4 - 168,000 + 150,000 - 318,000 

% 

I 

O 

21 - 0 4- 0 - 0 

c.., - 0 

22 - 360,000 + 0 360,000 

042-0 

23 * 0 + 0 » 0 

C43 - 0 

- 168,000 + 0 168,000 

45 - 

1 

0 


+ 0 
+ 0 
+ 300,000 
4 150,000 
+ 0 
0 4-0 
0 
0 


0 

0 

300.000 

150.000 
0 

0 

0 

0 


Matrix: 




660,000 510,000 468,000 318,000 

0 360,000 0 168,000 

0 0 300,000 150,000 

0 0 0 0 


Effejtlveneas Calculations 

Equation 3 still holds, and the D Matrix Is unchanged: 


Y dij c aj - (0.69)(660,000)4-(0.09)(510,000)4-(0.19)(468,000)4-(0.03)(S18,000)-599,760 
J-l 


£ c g j - (0) (0) + (0.78) (360,000) 4- (0) (0) 4- (0.22) (168,0001 - 317/760 

. 1-1 
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£ d^ 0 3J - (0) (0) + (0) (0) + (0.88) (300,000) + (0.12) (150,000) - 282,000 

J-1 


c 4j " * 0) * ^ (°) + (°> <°) + M " 0 

j-1 

559.760 

317.760 

and 5 - [0.48 0 0.32' 0.08 ) < 282,000 

. 0 . 

- 287,885 + 38,13’ + 90,240 + 0 - 416,256 bite - Answer 


6K271IK t' 

Pennants 

« inoreas« in 7 values results Ins T A - 18; - 36 

• The methodology Is identioal to that of Question 1. 

Availability Calculations 

Availability A « - O.69 

Availability g - ^ - 0.86 
A Matrix: A - [ O.59 0.10 0.27 0.04 ] 


Dependability Calculations 

Rellability A (3 hours) -.e 

Reliability,, (3 hours) - e 


• 1 

- • J 

i..- 

0.167 

-0.84 


• 1 

m © 

i-a- 

o‘. 083 

- 0.92 



"o.77 

0.07 

0.15 

0.01 


0 

0.84 

0 

0.16 

as 

0 

0 

0.92 

0.0? 


0 

0 

0 



t Matrix: I* 
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EXHIBIT X 


Capability Calculations 
Comment i 


All values, axoapt for and Og^, are Idantloal to those In 
Question 1. 

From Equation 1, c 2 ^ - 3.19 
From Equation 2, -0.56 


fi£ Matrix: C^ 


111 0.56 

010 0.19 

0 0 1 0 

0 0 0 0 J 


Effectiveness Calculations 
From Equation 3, 

B - f 0,59 0.10 0<?7 0.'4 ] x 

« 0.59 + 0.09 + 0.25 + 0 - 0.93 » 


. — 'ounded cl? 
r~ /-. from O .996 

1 1 
0.87 i 

0.92 I 

_ 0 J 

Answer 


MfSTMM 

Consent a 

• 5076 Increase In T vtluus results in: * 18; 36 

• A and D Matrices are those of Question 3 

• ®U Matrix methodology Is Identical to that of Question 2 


Capability Calculations 
From Equation 5, 


hi < 3 - ( 120 , 000 ) ( 1 . 5 ) - 180,000 

S^l < 3 “ (100,000) (1.5) » 150,000 


Matrix: 


660,000 510,000 
0 3 ^ 0,000 

0 0 

0 0 


480,010 

0 

300,000 

C 


330,000 
180,000 
150,000 j 
0 


] 


J 
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If feotlvenaaa Calculations 
from Bquation 3, 


a - [ 0.59 0.10 0.27 0.04 J x 


619,200 
331,200 
288,000 

L 0 J 

365,326 + 33,120 + 77,760 - 476,206 bits 


Answer 


HEtTHK I 

Ooansnts : 

• decrease in H values results in: 1? A - 4; Kg - 3 

• The only change from Question 1 ooours in the calculation of the A Matrix: 

• The methodology is identical to that of Question 1 


Availability Calculations 


Availability A - - 0.75 

Availability^ - §£ - 0.89 

A Matrix: A - ( 0.67 0.08 0.22 0.03 
Effectiveness Calculations 

B - ( 0.67 0.08 0.22 0.03 ) x 


0.99 

0.81 

0.88 

0 


- 0.92 - Answer 


SgtSTMl 

Concents 

• 5056 deorease in ft values results in: - 4; Kg - 3 

. The only change from Question 2 ooours in the evaluation of the A Matrix. 

• The methodology is identical to that of Question 2 

• The ^ Matrix f.s identical to that oaloulated in Question 5 


SXJGBIT 1 


amcc m-m 
KftftUwwM . Pfl mint tan* 


599,760 

S - [ 0.67 0,08 0.22 0.03 ] X [ 317,760 

262,000 

0 

401,839 ♦ 25,421 + 62,040 - 489.300 Hit* - An*wr 


*A 


*A 

h 

*X 

p( k 300,000) 

*2 

Avcrag* Blfsk 

12 

24 

8 

6 

0.86 

416,256 

18 

36 

8 

6 

0.93 

476,208 

12 

24 

- - 

4 

3 

0.92 

489,300 
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Th* method followed In any cost prediction is straightforward enough but it 
opt to bo quit# laborious. Furthermore, the date on which any prediction Bust be 
based are difficult to collect* and the gross estimates that it Is necessary to 
employ auot be treated with a good deal of reserve . To Bake any cost prediction 
at all* it is necessary (1) to break the expenditures down into rather snail 
categories* (2) to collect as much past experience on expenditures in each category 
as possible* and (3) to predict from this information how such is likely to be 
spent in each category for the project being costed. Thereafter* all the cate- 
goriee Bust again bs cosblned to obtain the systsa cost as s function of tine. 


2.11.2 The RAID Method 

Many agencies* both in the DoD and Industry, are performing military cost 
analysis and developing costing aethodology. The RAMS Corporation ’os been one 
of the leaders in the costing field. The aajor costing concepts proposed by the 
RAMS Corporation are ae follows: 

• Categorisation of coats into research and development, initial investment, 
and operating costs 


. Use of lraivldual-systea costing and total-force-p cructure costing 


• Use of incremental costing 
. Concentration on aost lap or tent cost factors 

These concepts ere described briefly below, and sevaral of them are discussed 
In wore detail in the following sections. 


The categorisation of costs into research end development, initial investment, 
end operating costs Is consistent with the Sop progressing system. Some advantages 
of this categorisation are that the tiae phasing of the costs are readily apparent, 
the total lifetime cost for alternative systsa lifetimes is easily obtained, and 
the Impact that changes in the research, development, and initial investment costs 
have on opo rating costs can be observed end traded off. 


Total-force-structure costing is much mors Involved than Individual-system 
costing. Xndlviduul~systea costing does not examine the Interactions between 
itself end other systmu in the total force. This makes the cost analyst's task 
simpler, and la particularly useful in costing future systems (where Interactions 
with other systems are not well defined anyway). Total-force-structure coating 
examines the cost of e system in the framework of the total force. This requires 
lnfomatlon on interactions among the syetaas in the total fores, and also cost 
data for the total force. 


AMCF 706"1*i 

Incremental coating la an approach that dvtsimlnes the change In coat 
associated wltn achieving aoaa change In effectiveness. If a dec la Ion la made 
today to develop a new system, Incremental coating la uaed to determine tha coat 
to develop that system starting from today. Coate that have been Incurred 
previously are not counted, and the coata for exietlng equipment and facllltlea 
that can be ut lilted In the nee ayataa are not counted. In moat esses, Incre- 
mental coating la the type of coatlr* that decision-makers are asking for whan 
they any, "What will X be getting for ay ncoey?" 

A eenelllvity analysis cai'<31ats or ehenglne each of the variables . in the 
study In turn, while the reaalnlng variable a are held constant, to determine how 
anall changes In each variable can affect the study reault. 

A cost-senaltlvlty analysis la normally uaed to determine which parameter* 
have the greatset impact on the total coet. The cost analyst -can then concentrate 
hie efforts on the most Important coat factors. 

2.11.2.1 Coet Categorization 

System costs ha-e been categorized Ir a number cf ways, depending to a grea» 
extent on the type and applicability of available data. The objective In an; of 
these categorizations la to focus attention on the major resources that will be 
consigned during the life of the system. Information on resources le produced 
that can be compared with Information on available resources; alternative courses 
of action can be evaluated according to the amount of resource consumption they 
involve . 

The military grouping of costa corresponds to the program phases In which 
the costs are Incurred: 

, Research and Dev e lopment costs . All the coats necessary to bring a system 
Into readiness for introduction Into active Inventory. 

. initial Investment Costs . All costs Incurred In phasing a system Into the 
operational force. They Include the coate of procurement of prime and 
apodal equipment, facility construct Ion, personnel training, and procure- 
ment of Initial sparse. 

* Or e rat Ira Coat* . All costs necessary to the operation of the system once 
it has been phased Into the operational inventory. While both RAO and 
Investment costs are Incurred just ones, the operating costa continue 
throughout tie life of the system. 

The curves of Figure 2-8 show typical distributions of thee# ooats ever the 
life cycle of a system. Further subdivisions of these costs are shown In the 
followlrg paragraphs. 
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Examples of ths typos of cotta In each aajor category are at follows: 
Rete arch and Develoc— nt Cotta 

• Dealgn and development 

PrelUilnary rate arch and dealgn atudlea 

Development engineering and hardware fabrication 

Development lnatrumentaclon 

Captive teat operations 

Fuels, propellant a, and gates 

Industrial facilities 

• System teat 

Teat-vehicle fabrication 

Vehicle spares 

Teat operations 

Teat ground support equipment 

Teat facilities 

Test instrumentation 

Fuels, propellants, and gases 

Data reduction and analysis 

Maintenance, supply, miscellaneous 

• System management and technical direction 

Mi*&L JESSS3SSSL M! 

. Installations 

Construction of new building, airfields, etc. 


M5 
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• Iqulpment 

Primary-mission equipment 
Specialised equipment 
Other equipment 

• Stock*. 

Initial allowances 
Maintenance float 
equipment spares and spare parts 
Combat consumption stocks 
Abb unit Ion 

• Initial tralnli^ 

• Miscellaneous Investment 

Initial transportation of equipment and spares 
Initial travel 

Initial propellants, oils, and lubricants 
Ope retina Costs 

• Equipment and Installations replacement 

Primary-mission equipts nt 
Specialised equipment 
Other equipment 
Installations 

< Tralnlnq 

• Pay and allowances 

. Propellants, oils, and lubricants 

Primary mission equipment 

Other propellants, oils, and ii&rleaats 

• Services and eiscellansous 

Transportation 

Travel 

Other services and miscellaneous 

• Bond tract adelnistratlv* and support costs 

2.U.2.T Coes ins ane Uiduel ftHgl %nd total Pores jtr^taare 

A ailitary *.-"*?<*» ean normally be defines by asacrlblng tarse key eljmenUt 
(1) the alee ton, with threat and environment} (?) the method of operation} and 
<3) a description of the physical makeup of the lystem and its support system. 


»-A6 
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If tease tirM iLwdU are defined, and there are no lnterrslat ions hips 
Kith otter qrstau ttet aunt te eonaldarad, ttea tte iypa of coot analysis that 
ti — — ***y used to tte Individual iyit«>cott analysis. 

Individual systte-oost analysis la lost Involved and raquiraa fever data 
tten mild te n eceaaary if tte systan war* not laolatad froa tte total forca . 

tte difference betwe en lnfivldual-systea and total- force- strut turn cost 
analjaaa is tte levsl at abltli tte ana Ira If la carried oat. tte total-forca- 
atraatora coat analjra' la a hlghMr-levsl analysis and includes many individual 
eye tan with tte aaaoelatad lotaractloaa s n o ng tea Individual systeas. 

- — - uaaa* in afrteh an teirtte It qrtwi cannot te coated realistically 

without tte uaa of a total- farca-atructura coat analjra la. For aua{ Is , tte 
development of a now anti-aircraft w ea pon would require a total- force-stiucture 
coat analjra la. tte entire air-defenoe c ap ab ility could te coated (1) with, and 
(2) wltteut tee new Individual eystea, tte difference in coat between (1) and 
(2) would te the coat of tea new anti-aircraft w eapon. 

2.H.2.3 issaisstakSstsZiai 

Incraaa nt el coating aeeoonta for additional coats associated with the 
additional eff active nees of a new ayataw. tte aajor factors .ncltted in tte 
concept of Incrnaental coating are liter it ad assets, sunk coats, and salvage 
value. 

Inherited assets are those existing equipments, eria^lng facilities, and 
tralnad personnel ttet are em liable for tte as sys&te. Inherited uhU are 
not Included in tee coat caaparlson for alternative ayataaa. For example, it a 
new radar aystan can utilise existing repair inetallationa, the Initial coat of 
these Installations is not inclu d ed in tte coat analysis; i.e., tte installations 
are free. 

gunk costa are tease cotta that tears been expended prior to a given decision 
pc'ftt, in ttas, and tease coats are not included in a cost Cteparlaon. For 
axaapls, wppoaa two alternative esamnleatioas syatewe ere Ming considered for 
di I'slnpasnt, ant one eyataa has already I n c ur red $a million of HO funds, while 
far tte otter alternative eystea (possibly an Improved rareioe of an existing 
ays tea) no BtP fund has been expended. Tte dec is lor to select either alternative 
any be baaed naturally on coat, effectiveness, technical frees ibillty, time 
sc hedul ing, etc. Ncwever, in coating each alternative, the #2 million that has 
already barn spent on tte first alternative is a sunk coat, and therefore is not 
included la tte coat scaparteoa — i.e. , no natter which alternative is selected, 
«a additions! $1 minis* of MB ft jnd* will be eapeoded. 
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Another factor included ta the lcgrsasntal costing concept la salvage vtlm. 
Salvage value takes into account the cost saving that uj roallaad tree s suing 
or traneferrltg a systsa to a future organisation wh nr the ext tea la phased oat, 
or the coat aarlag free aalllng the sjratea as scrag. 

2.U.3 Coat-Kstlaatlna Isiat lane hies 

The aost Important tool available it the coat analyst la the coet-eethaatlng 
relationship (CAR). These provide a netted of prslictlng the cost of a no* 
systae. Thee*, relationships are developed by collecting eoctesulated cast dots 
on alallar systeea and correlating each costs to eppr o pr late character let lea of 
the new systae (weight, slat, nafesr of parte, etc . / . tar sane types of syeteee 
the*' ?*vo non a in existence for eany years, each as elrfrenee, enough dt.*a have 
been generated to penult deve l ops id of CBa for use on now alrfiaeee; ho wev er , 
for sore advanced eyataae that incorporate • substantial amber of state-of-the- 
art taproveaaots, currently available etat-pr edict ion techniques do not provide 
the needed accuracy. 

The standard wetted for developing coet-oattnatlqg relatlooehlpe is through 
the use of aultiplc-regresslor analysis. To use the aultlple-regreeelon -pp roach, 
e general essuwptlon le aade that the dependent variable — in this ease, a coat 
category — le related to the predictor variables by a linear equation of the 
following fonet 


f o* C ) " V + fc l f i**ll» x 12* * • *» *r x l) + 2 * 21 * 12 ’ * 12 ’ 

* * ' - * b n r n( x ln' *2n’ * * *» 


*' *t ? 2> 


share 


f D (C) le e function of the coat 




le the J th prediction paraaeter In the C th subset 


f ere func tiers of the x’e 

n 


b. 


b ere eaapvted regress ice cocff 1c tents 


Although the general function le llruar with respect to tie regression 
coefficients, it le not necessarily linear with r esp ect to C or 'he x's. Ttus, 
f >r axaeple, an equation of the form 

c • m x l * e * * 
trans forts to the required ltrear font 

log C 


log e . X> i let X x * bj Xg 


Altteif': nonlinear equation* can he used through various netheeetleal curve- 
fitting procedures, the advantage* of s* log a least-squares analysis b aae d on 
linear equation.* in the coefficient* erd the flvsltlllty of weft linear fores 
straggly fa vc. * the standard regression approach. 
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lamlly, M;mt* CKi art developed for each of tut aajor coat, categories 
(lMMteH and Development, Initial Investment, ana Operating Coata). factor* that 
lnflaanoa one coat category Mr ten littia effect on anot.h*r '■•♦•^sry or ear wan 
km ea mai lt* affaet. Par uwapla, an equljacnt itre-higb-reliatolllty prog rat, 
sue* an that currently being s po ns ored bjr DoS through the three ivrvlsti, will 
1 nereaee dnwalflpMnt costa bat# It In h o ped , greet ly red ucs op e ra tional ccsv*. 
therefor*, paras at are relating to reliability will km a positive developsent- 
eost relationship bat e napatlve operational -coat relationship. it would be 
marine to oaablne tea two coat categories into ooa cm since accuracy will probably 
be dla ini abed) erpally important, trade-off aapacte will be lost. 

It la dsslrabls tc Halt the final oalb a r of parameters in the :®e to 
alirttetaa use difficulty as wall aa to cexlalse the degrees of fieedea to tha 
analysis. 

2.U.4 aaiatetii 

2.11.4.1 Oat C' ^ eseneurabllltr 

Tbar* are tee aaln techniques far ankln» costa ccassemurable: sear tl tat ion 
and discounting, A mo rtisat ion la the spreading of tha aystaa research and develop- 
aant red initial i nv a a ta s nt coats over tbs 1 if atlas of tha aystaa. Mscounticg 
13 aoad to reflect tha greater value of pr aa a n t a easy ever future isoaey, because 
of the possibility of invest lng pr a aa nt sonsy for a pain. 

b a ^rt 1 sat Ion and discounting are not mraal l y need for allltary coating# 
far the fallowing resa-ss: 

• ftoaarnaent directives call for yearly astteatea of actual expenditures; 
this inf oraat loo aids In the pr spa ration of the animal budget . 

• the d i scou nt ing rata Is i lfrlcs.lt to determine; it baa been estimated to 
be free 1 percent to 20 percent. 

• these factors are noraally inaenattlva aa coapered vita other costing 
coast drat lone. 

At pr sent, tha discounting rate nameHy used la Amy studies la 15 percent. 

2.11.4.2 Poet Oneertalotv 

l e er y aoet eettaate la aaeertela# trim tr, initial ooaposeat-coat aattaata 
«p tkreugb its aggrapatloc of ayvtaa coats. Coat analysts have base careful to 
differentiate between this type of uneerta nty in the eoet eettaatws (coat- 
eat beat log uncertainty) and uncertain! ice m skat exactly u to be eeatef (require- 
oenta uncertainty) . Ia the let tar cats, e detailed description of tha inter- 
relationships associated with the eye tea any ernawsi a* each as 75 percent of the 
total project tie#, tha aoet a«rtau errors la tea coat analysts can usually be 
traced to tna wept loos and inter relationships «*a rfdu tbs coat estimates are 
tewed. 
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Cost-estimating uncertainty la tha statistical uncertainty ceased by irrori 
in the cost data, inaccurate cost-estimating relationships, and differences ir 
the cost-analysis approach. 

The analyst should dsvslop not only the cost estimates, but also an indication 
of the confidence level or possible ranees of tha costs. Use of the coet-aenel- 
t tv l tv analysis is also desirable to show the impact that uncertainty Van have on 
the final cost estlaales. 

2.11.4.3 Data Collection 

One of the greatest problsas In cost analysis is obtaining sufficient data 
and accurate data. Tha basic data ccnpiled to support the re^lreoenss of the 
cost analysis should aset the following tHifulrmantst 

. They should be collected in sufficient quantity to provide significant 
sample sites of the various system characteristics and cost parameters 
being etudled. The confidence in results increases with the quantity of 
observations. Accordingly, every effort must be made to acquire sufficient 
data fron actual surveillance of systems In an operational environment. 

If, however, adequate data of this type are not available, it may be 
necessary to resort to estimating techniques. Several proven techniques 
are available for various equipments. 

. They should reflect current system conditions. Timely collection of input 
data Is required If the cost analysis Is to depict current conditions in 
the system. Many diverse factors affect the cost of developing, purchasing, 
operating, and maintaining a modern system. Unfortunately, at least from 
a cost-development standpoint, almost all of these factors are dynamic. 

. They should be accurate. The Importance of using the moat accurate data 
available cannot be overemphasised. The stringent requirement for accurate 
data Is related to the intrinsic nature of the uatheoatlcsl approach. Many 
compound summing operations (or multiplications) will he accomplished 
during the cost analysis; thus any Inaccuracies in the data Jill *>l«o be 
compounded during tltese mathematical manipulations. 

. They should be representative of the operational situation of interest. 

The system-cost characteristics ere known to he affected by .he operational 
aid maintenance envlrormant . Until such time as tha direction and extant 
of the various Influencing factors upon the system are more explicit!? 
defined. It will be desirable to collect data frem the specific operational 
situation in the cost analysis. 


2-50 


AMCP 70S-181 


2. XI. 5 Coa>Aflalf <« flg£llg£tton 

2. 31. 5.1 Froblma ffermulmt Aon # 

The spec if 1c font that ary cost, aixlysts will take depends on the particular 
sjr titan be Inf studied. However, in general, there are three factors that must be 
considered In every casting problea: (1) the costing method* that will be 
employed, (2) the type of data that will be required, and (3) the sources of the 
data. These factors ars discussed below. 

The costing method employed le normally one of three general types -- a 
catalogue price, a cost-estimating relationship, or an estimate based on a similar 
system. The catalogue price : ls used where the component le an off-the-shelf item. 
The cost-estimating relationship can be.develeped on the basis of design and per- 
formance characteristics, or previous cost. The estimate of cost based on a 
similar system is used when the system being analysed Is sufficiently similar to 
an existing system that a valid cost analogy can be made . 

The type of data that is required for a cost analysis may be categorized into 
the assumptions and constraints, the description of the system, and the cost In- 
formation. As an example of the types of data required for a cost analysis, assume 
that the avionics on a group of helicopters is maintained by a dual maintenance 
organization, consisting of organizational maintenance and direct support mainte- 
nance. 

If a complaint against an avionic system Is received, organizational personnel 
try to verify th* ccmp’aint; if they verify it, they (1) perform seme maintenance 
at tSe helicopter (this may consist of Changing a black box) and (2) in a certain 
percentage of the cases, generate some direct support maintenance. 

If they do not verify the complaint, they have, of course, spent seme time 
in the investigation; however, that complaint Is disposed of. 

The following data, ars required 'r, the cost analysis of this system: 

• Number of complaints per month 

• fraction verified 

. Manpower needed to verify 

. Fraction disposed of at the helicopter 

• Manpower needed to dispose of the complaint at the helicopter 

. Cost of the bite and pieces needed at the helicopter 

■ Manpower needed to provide the bits and pieces at the helicopter 

. Marqmwer needed to replace the black box 

. Cost of the replacement black box 

"•From ^KTLTaHTTFy Engineering", copyrighted 1964 by ABINC Research Corporation, 
Publisher - Prentice-Hall, Inc., Er^lewood Cliffs, New Jersey. 
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• Manpower needed to obtain tha blaok box 

• Tina da lay before tha faulty bos ia repaired and ready for uaa 

• Tlae needed to repair tha black box at d irect^ewpport 

• Coat of the bite and pieces needed at direct support 

• Manpower needed to provide the bits and paloaa at diraet aup p ort 

. Coat of loadad manpower for various categories of diraet labor 

• Average llfatlaa of a black box 

• Cost of various categories of loaded manpower 

Sources of data Include published reports (and unpublished back-up material)* 
equipment catalogues* and financial swear lee. Typical sources of data for the 
example given previously are Indicated below. 

The number of complaints per month, g 

The quantity N is composed of usage rate* reliability* and the number of 
black boxes in use. If n black boxes are being used* each an average of t hours a 
month* and if the complaint reliability, l.e., the mean tlae between complaints 
(MTBC) on the black box. Is > c complaints per hour of use* then / 

» at 

H - r- 

A c 

The MTBC can be estimated from field-failure data. 

The usage rate* t* must be estimated from deployment plans* as must the 
nvetber* n* of boxes in use. On a projected system, all these factors will be 
available. Hence N can be estimated. 

The fraction of verified complaints 

The fraction of verified complaints, v, is another output of the observation 
of field failure data. If is the mean time between verified complaints* then* 


The fraction of verified complaints that can be disposed of at 
the helicopter 

The fraction of verified comp Hints that oan be disposed of at the 
helloopter, f, Is estimated essentially from two pieces of informations 

(1) A maintenance plan that defines the repairs that will be made at tha 
helloopter. (Adjustments, for instenoe* will often be ma«»* there.) 

(2) A reliability prediction in greater detail than those needed for and 
\f , namely, a breakdown of A into thoee oases which will be disposed 
of at the helicopter and thoae which will have to go to direct support . 
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Such prediction* can U obtained fro* * acre detailed knowledge of the 
equipment; or, alternately, etatlstlcal value* of the fraction f for 
slallar equipments now In eervlce can he ueed aa eatlaatea. 

the coat of maintenance manpower 

Direct-maintenance manpower la Obtainable from maintainability prediction*. 
Suppose the direet-melnte nance time required and the corresponding hourly pay 
required on the average to accomplish the maintenance act lone are, respectively. 


where the subscript 1 refers to the skill class. Then t^ can be obtained fro* 
maintainability predictions; can be obtained frem Hats of pay classes, together 
with an estimate of the useful life of maintenance men in grade . Here a suitable 
definition of useful life might be the percentage of the time in grade during 
which the man is actually assigned to maintenance duties . 


Bee idee direct labor, there la In any organisation a great deal of overhead 
labor. Much of this is concerned with scheduling and supervision, and some with 
management; mod a good deal goes to leave, training, and nc maintenance duties 
of ths men thmsselvee. 

In general, the loaded time will be e linear function of the direct-labor 
time. If Tj^ le the loaded time (direct and overhe a d) spent in labor class 1, then 

O T i “ *1 + Vi 

The cost of labor is then given by 

C(T > 'I *»°« ■ l *i«i * l ».»»«. 


The constants Involved In the equation above are the overhead coefficients, 
a t and b^. Rough estimates of these can be made from tables of organisations 
and frrm estimated work-loads. 

A good approximation to the equation above can be obtained in the form 

C(T) - C a + .t 

where t le the total active-repair time in all labor clesees. 

The materials costs at supply 

The materials cos*s at supply are the parameters that connect the different 
echelons. To estimate them, two kind* of information are needed: (1) the average 
•mount end kinds of materials needed to perform repairs, end (2) the cost of these 
materials to supply. These categories are diecueeed below. 
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Estimates of ths kind* and quant it la* of materials can bt obtatnea either 
fra* * statistical analysis of the behavior of s la liar equipments la prase nt «h 
or from a ds tailed reliability ualysli, based or. actual sc bimat ice. 

Ths cost of these aaterlals at the supply echelon In question will consist 
of ths following! 

• Cost at the higher supply echelon, whi-h supplies ths one in question 

• Cost of ths labor needed at the higher echelon and at the echelon la 
question to move ths matarial* 

• Cost of transportation 

The materials coat at one echelon then contains implicitly the accrued supply 
costs at all higher echelons} thus the whole system-support cost will accounts* 
for. 

The cost of supply manpower 

The labor cost at supply mist be obtaluu by an analysis similar to that 
described for the maintenance manpower. If a detailed analysis is not available, 
probably the bast aetlmate obtainable is to assume that every action, i.e., every 
requisition and every issue, takes, on the average, about aa much labor as every 
other. Then if the total payroll of the supply organisation le divided by the 
number of pieces of paper generated, an estimate of ths labor cost of requisitions 
and issue is obtained. 

The time daiay; the average life of a black bos 

The cost of time delay {involving AT, L, and C Q — the time delay, the 
average life of a black box in the (partial) system, and the cost of the black 


Time delays smst be estimated by 
The average life can be estimated 

echelons of maintenance, and the 

condemned, and g aro returned to 
if there are n boxes in circula- 


te In | and b can be estimated from a detailed reliability analysis and a 
maintenance plan, or from statistical values for similar equipment. The number 
n Is determined by ths verified failure rate and by logistic policy. 
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box to supply, respectively] has been given as 


& 


The estimation of C has been discussed above, 
c 

observations on similar support organisations, 
from condemnation rates, return rates to higher 
total number in circulation. 


If on an average a black boxes a month are 
higher echelons of ualntenanse for repair, then 
tlon. 
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2.11 tMHISI 1H HKi& 

2 . 12.1 SssSSSi 

Th* Method used la were la ln« th# nodal if dependant on tha type and coap lax- 
ity of nodal and tha tim, equlpmnt, personnel, and noney available. 

for a complex nodal, tha tine and coat Involved In exercising tha siodal on 
a computer nay ba nueh laaa than tha tine and coat for using people with elida 
rulttf . However, If thla nodal la only required to ba axerelaad once, tha coat 
of aet-up tine for tha conputar nay preclur* Ita use.* to general, th* computer 
la noch faatar for exercising a nodal than using a slid# rule or a dealt calculator 
coca tha nodal haa bean programed, to actual practice, hewavar, tha analyst has 
an overall tine constraint on tha study effort, and If a conputar la available, 
tha tine to exereiae tha nodal will not change, because tha analyst will use any 
additional tine to conduct eeneltivlty analyse* or axpar.d the orlglnrl nodal. 

Th* basic point to ba nad* la exercising tha nodal is that no natter what 
process la used — desk calculator, conputar, or slide rule — th* final result 
will b* only as good a* tha nodal and tha Information put into the nodal. The 
use of a cosputar does not in lteelf ensure a more valid result . 

2.12.2 Analysis of Output Data 

than an analytical nodal has been developed and sufficient input data gathered, 
the nodel can b* exercised, either mousily or by naans of cosputar. to the ala- 
pleat of easea, a single dependent variable will .vault fron th* process, to nost 
system analyses, however, a whole f sally of dependant variables will be generated. 

2.12.2.1 A nalysis of a Single Dependent Variable 

Each aodel equation will yield one output paraneter (the dependent variable) 
when one set of input parameters (independent variables) is used. The output 
paraneter night represent er average, predicted, or eetinated value. The value 
could represent a mature of coat, effectiveness, reliability or any other paraw- 
eter of interest upon which the wousl was baaed. 

The single output paramter could of course be analyzed by ecw^ariis it to 
•am previously known standard of acceptability, for axaaple the objective of the 
analysis nay havz been to eatimte tha i si lability of a product to determine com- 
pliance with a pre-established rsqul ramn t . to this instance, a judgnent of 
acceptability of the product night be wade by siaply canparing the eatlmted 
value with the required value. 

to nany eases, however, it is desirable (and often necessary) to analyse 
the resultant fron tha standpoint of tha associated uncertainties. 

____________ * 

* Ibis ia on* area in which a coat-effectiveness study could be used to dvter- 
nlne Aether a coapater should be eaployed. 
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Will* often difficult to adult, various re — on* contribute to uncertainties 
which affect the nature of estiaate*, asaeejraaaate, or expected outen— « . Bum 
uncortalntiea ere not necessarily indicative of faulty or caroloss Mttudln 
proceduree, hut rather are reflective ef the difficulty la ehereeterlalat the 
real world, free previoua experience ctnple ifldeeM aditi that the proeeee of 
prediction la a difficult eat. Bitter? telle ue that the satlaatsd eoet of build* 
ism the Suet Canal me off hr a factor of twenty. Coot eetiaatee for recently 
built nuclear power plenta have been half of uhat they eventually wo un d up met* 
ms. 

in 9 out of 88 aajor hardware tyttma studied recently by the aanf Co^pciv- 
tlon, the actual procu r w etn t eoet per article was acre than three tiaee an early 
eetiaete of eoet} end in two caaee the actual mat was wore than ten tiaee an 
early estiaate. 

The problem of uncertainty in eetiaatlni coat and affect ivenrrn for nit eye* 
teae cert of course, never bo eliaiaeted. However, the eoalytt can ainiaixe un- 
certainty and — aore UportMt — can account for it in providing inforaaticn 
to the declelan-uaker. This, of couree, requires knowledge of the types of 
uncertainty which night be encountered. 

Charles Hitch and Roland McKean, in their book. The Bccnonlca of Defame in 
the Hucloer due. (Barvtrd university Praes, 1903) describe five basic type# of 
uncertainty associated with eetiaatee i 

(a) uncertainty about planning and seat factors 

(b) uncertainty about strategic context 

(c) Technological uncertainty 

(d) uncertainty about she may and hie reactions 

(e) Statistical uncertainty 

The analyst la confronted with the teak of deciding how uncertainties are to 
be treated. The neat laportant advice is of couree, "Don't ignore than*, secondly 
he auat be able to recognise the type of uncertainty involved. Third, he met be 
able to distinguish between the laportant and unlaportent uncertainties in eon- 
text with the particular analysis, finally, he auet be able to expend on Ria 
basic eetiaete or aeoauraa en t by additional consideration of the contingencies 
created by the uncertainty. This my entails 

(a) expressing the dependent variable ee a range of valuna, each value 
having e probability of occurrence. 

(t) Assigning confidence intervals about the estiaate. 

(e) Subjectively Qualifying the nature of the estlasted or aaaaurod 
value. 

Figure 8.9 illustrates six different ways of expressing an estlasted value. 
Sach successive express 'on fora represents e higher degree of specificity in 
treating uncertainty. 
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EXPRESSION FORM 


DEGREE OF SPECIFICITY 


1. System A la estimated to cost $15M. 

2. System A is estimated to coat |15M) 
however, the analyst Is not sure 
(uncertain) about the figure. 

3. System A Is estimated to cost 
between $11M and $19M. 


4. There Is a "strong probability" 
that System A' a cost will be: 
$11M - $15M - $19M. The $15M Is 
some measure of central tendency 
(mean, node or mean). The $11M 
and $19M are the estimated lower 
and upper cost limits. 


5. 


6 


With a .95 probability. System 
A's coat Is uotlmated: $11M - 
$15M - $19M. The numerical 
expressions have the same meaning 
as In 4 above. 



1. No uncertainty expression. 

2. A vague qualitative expression 
of vuvsertalnty Is given. 


3* A range la given to express the 
magnitude of uncertainty. How- 
ever, no probability information 
Is given} It Is not stated 
whether the analyst believes 
there Is a 1%, a 10# or a 100# 
chance that the cost will fall 
between $11M and $19M, nor Is 
It Indicated whether the cost 
Is likely to be closer to $11M 
or $19M. 

4. An adjective descriptor is added 
to convey a rough indication of 
probability. 


5. The adjective descriptor Is 

replaced by the more definitive 
rumeral . 


6. A complete probability distribu- 
tion Is given, and this is 
depicted by a curve. (Both the 
pioblems in getting the Case 6 
type information and the amount 
of additional information pro- 
vided by Case 6 are of a greater 
magnitude vs Case 5 than Case 5 
Is vs Case 4. Case 4 is vs 
Case 3, etc.) 
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2.12.2.2 An alysis of Several Dependent Variables 


Most systems analysis problems encountered will Involve the treatment of 
more than one dependent variable. For example, quite often two alternative 
systems or alternative designs are the subject of the analysis. Further, at 
least two dependent parameters, — e.g., ccst and effectiveness -- (and probably 
many more) are of Interest in making the comparison. An even more complex sit- 
uation arises when the study objectives Involve "trade-offs" where the dependent 
vrriables of Interest can assume a broad range of values. In general, the task 
of analyzing data outputs can be subdivided into: 

•Comparative’ Analyses 
'General Trade-off Studies 


2-57 



AMC’Kt'ttt 


CMMirttiw imItim, as daf! -ed herein, in iMIn >1—1 tt Mtmtaiii 
which of two or more alternative* id better (or hoot). Jf • dlafclu lipsMdmat 
variable constitute* tho basis for covvImbi tin analysis is a ro thor ihfU 
on*. Bence, the problaat night bat 

’ which of two eystane i* noro effective* or 

* Which of two ifitowi i» ch e a p er* or 

* Which of two syetsno it c h eap er and better? 

m t rooting tho above tjrpH of problems, tho analyst 1 # r espo n sibility io to 
recaamend • single eholeo from mo at tho alternatives. Little la 10 ft for tho 
decision -aabtr, except to loeo^t or reject tho laoult* of tho analjrais. 

In many cut*, however, tho analyst ‘e objective ie to present mart than one 
altemetive to the declaloo-aaker. For exa^le, tho tamo of tho analysis nay 
require that separate values sf coat and sf i e.t lva n s s * bo p r oa a nt ad for each 
ayotan considered, with final choice of tho “better one* being tho decision- 
aeker't choice. An exaagle of this approach, given bolow, above that systen B 
coats aorc that ayetan A — but U also tore affective, in this axaaple, the 
Judgment of the relative worth of tho two ayatene could, by intout, be that of 
the decialon-aaker. 

ses25 £ss* 

A ll.OM .99 

B 91.5N .9f 

A similar axaaple is one vnereln *he decision-maker desires to subjectively 
consider certain factors wnlcb, by intent, have not been uKluueo in the model, 
t these factor* .. t called "leverage effects* and will be discussed In greater 
detail In Section 2.1?.) An txeatple of auch a case la evident io the analytical 
process recently employed to select a Chief of Police for the Citv of let Angelea 
In that situation, there won sarerrl candidate* frr the position. By means of 
a vlgo rev* analytical approach (actually a nodal wan developed with which to 
set lusts the "effectiveness* of each candidate), the maker of choices was re- 
duced to three, the Ultimate ejection of the beat mar. for the Job woo aode 
by choice of the dec 1* ion-maker <la this case, the top city offlciala). m the 
example cited, it la to be noted that the analytical model was used to alainlse 
the number of possible alternatives, but au not neceeaerlly used to arrive at 
the final decision. 

Often, the objective of the analyst la to conduct t rede-off studies where 
tv© or more dependent variables are to b* considered over a breed range of 
(Kwelbliltie*. thl* font of objective la quite common during tho concept form*- 
let lor yftoee of e pro. ret when rsqul r a o r ta are being developed, qp this situa- 
tion, the analyst has boat given a alnlw level of acceptability mi a design 
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Seal for th* dependent varlablss of interest. He than must aralyxe th« many 
altemsfci'.'ea within th* given envelop* In order to reduce the number of choice* 
to one — or to rome minimum number. Trade-off* can Involve weighing one per- 
formance parameter a*aln*t another or weighing performance against coat. Kxse^le* 
of each situation are given below: 

(X) Performance Parameter Trade-off* 

One of the major parameters in -wet effectiveness models is avail- 
ability. Availability is in turn a function of reliability and maintain- 
ability. When reliability la expreaaad at a frequency of failure (MTBP) 
and maintainability expreaaad as the length of time required to restore 
(MOT) a failed item, availability in some cate* can be expressed as 

A - hthF+t© t 

If a fixed level of availability la the desired output, it la apparent 
that MTBP and MOT can ba traded off in achieving the desired value. 

(2) Performance Vevaua Coat Trada-offa 

Another form of trade-off problem involves performance versus cost. 

In this type of trade-off vroblem the principle Objective ia to weigh 
varying levels of performance against varying levels of cost. Common 
trada-offa in this category include: 

* Speed versus coat 

* Payload versus coat 

* Reliability versus coat 

Considerable emphasis has been directed recently to the consideration 
of total life-cycle costa whan formulating system level decisions. The 
reasoning behind the emphasis is that incomplete consideration of the 
influencing factors oftan can lead to erroneous decision. A case in 
point can ba illustrated by the following example. 

It ia assumed that the decision-maker must choose between twe systems 
of differing availability on the basis of cost. It may be 
g«:. 'rally shown that development cost and initial investment 
cost increase with increase reliability (see Figure 2.10a). 

On the other hand »nnual operating costa (and hence total recur- 
ring costs) decr«M~e with Increased reliability (see Figure 
2.10b). It readily bocomes apparent that the cheaper system 
can only be determined by combined consideration of all 
coot* (Figure 2.10c). 
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The interdependencies between trada-c^f studies end the above objectives 
are shown in Pleura 2.11. Any trade-off study oust be based on a set of 
depend.mt and Independent variables. The status of a given proje^c In .eras 
of accomplishment of the objectives would identify the dependent and inde- 
pendent variables for the trada-ofr study effort. For evampl**, if require- 
ments envelop*® have been established, these constitute the set of indepen- 
dent variables for the trade-of* study. The elective of the study might 
then be the selection of a minimum set of design alternatives. 
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An Important tool in performing trade-off studies la the analysis of 
sensitivity. In general, sensitivity analysis involves determining tbs 
significance of s given variable within some prescribed range of interest. 
A simple illustration of sensitivity la given In figure 2-12, which des- 
cribes the relationship between total system cost and the endurance for 
a hypothetical manned aircraft. Bie analyst would conclude from the curve 
that : 

(a) Cost is highly sensitive to endurance at low levels of endurance. 

(b) Cost is relatively insensitive to endurance of higher values of 
endurance. 
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2.U BEVELQP lECISItN KtiL 


3 1 imi z at ion Criterion 


In defining an optimizing criterion, the system analyst is faced with a 
problem similar to that of putting in precise, quantifiable terms the rules 
or criteria for choosing the "best" painting or "‘bast" automobile. These ex- 
ample? do have quantifiable characteristics, such as the size of the painting 
or cos. of che automobile; however, artistic judgment and user experience, 
respectively, are factors in the final choice, m the same sense,, the choice 
of che b*ist system is greatly influenced by the use of good engineering, eco- 
nomic , and operational Judgment . 
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Xt it BMt important, however, that the optimising criterion be defined to 
the maxim extant possible, for the following reasons: 

(a) The inputs provided to the analyst through use of the criterion 
can reduce the else of the problem to a point where a judicious 
choice can be made. 

(b) Defining e criterion forces the analyst to examine all possible 
alternatives in an objective manner so that the criterion can be 
adapted to mathematical representation and analysis. 

(c) Xt la easier to incorporate the ideas and axper lance of others 
if e formal basis fdr optimisation Is established, 

(d) The (partial) basis for final choice is in precise, quantifiable 
term end can therefore be reviewed and revised, end can provide 
inputs to a learning -process for future optimization problems. 

When a criterion for optimisation' la being formulated, the system and the 
boundaries must ue explicitly defined. This definition will influence the 
choice of paramters in the optimization model. The purchaser of a new auto- 
mobile, for example, may or may not consider the service policies cf the manu- 
facturer and dealer. If he dees, the system is both the automobile and service 
policies^ if he does not, the myatesr is only the automobile. In attempting to 
optimize a weapon system such as a bomber, the analyst has to consider whether 
the system is to be defined as a single bother, a squadron of bombers, or the 
complete bomber fleet. It. is possible timt optimizing with respect to a single 
plane (a sub-optimization) may not yield the optimum "squadron" system. 

As part of the system-definition process, the analyst also determines the 
fixed and variable factors pertinent to the system. This task requires % pre- 
liminary analysis, since consideration of all pof^ible alternatives will usually 
lead to problems of unmanageable size. Seme factors may be considered fixed if 
results of previous analyses, perhaps sub-optimizations, indicate the values 
that have attained the best results in the past. Hi o maintenance troubleshoot- 
ing routine, for example, might normally be considered as a variable factor, 
but past research in this area mey be used to select a particular routine appli- 
cable to the systm under ctudy, or perhaps to restrict the range to several 
alternatives . 

Once the mission profile is defined, consideration can be given to the 
physical and economic limitations that will have to be imposed. These limita- 
tions are based on requireuenva and availabilities, and may involve such factors 
as minimum system output, maximum reliability, maximum development time, maximum 
weight and volume, and type and number of support and operational personnel. 
Through such consideration and snvelope of design, development, operational, 
and support alternatives can be established in such a way that each overall 
configuration within the envelope will meet physical and economic limitations 
as well as minimum performance goals. 
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Mow the analyst must select a decision criterion by specifying the types of 
effectiveness and cost paraneters to be investigated and by assigning numerical 
values where required. Thj choice of objectives and criteria is perhaps the 
most difficult task in system effectiveness optimization, it is expected, how- 
ever, that current research in the optimising of system effectiveness will 
develop theory and accumulate experience to help overcome acme of the difficul- 
ties of this task. 

It would be Impossible to establish rigid ground rules or procedures for 
formulating a criterion for optimizing system effectiveness. The answers to 
the following two basic questions, however, will provide a great deal of in- 
sight for such formulation: 

(1) Why Is the system being developed? 

(2) What physical and economic limitations exxstt 

The answer to the first question essentially defines the mission profile of 
the system. Where possible, the definition chould bo translated into quantita- 
tive parameters — a difficult task in many cases. A performance measure such 
as kill-probability for a SAC bomber may be assignable, but the bomber may also 
have a mission to act as a deterrent — a measure that is difficult, if not 
impossible, to quantify. It is for this type of multi-mission case that Judg- 
ment will become especially important. Even if quantitative requirements can 
be placed on all mission types, weighting factors would have to be Introduced 
to quantify the relative importance of each mission. 

Factors that have relatively little impact on overall effectiveness or cost 
can be considered to be fixed or, possibly, can be Ignored. There is, of course, 
a risk involved if factors chosen to be fixed or unimportsnt would have had a 
significant effect if they had been allowed to vary. Factors that fall in this 
"gray area" may have constraints imposed upon them in such a manner that the 
more detailed analysis to be performed in the optimization process will Indi- 
cate final disposition. For example. If a questionable factor might have a 
monotonic Influence on effectiveness, consideration of only extreme values 
might be all that is necessary to determine the significance of this influence. 

It is important that factor selection, variability, and the final choice of 
system definition be clearly indicated so that the scope of the optimization 
process will be known and areas for possible modification of the formal mathe- 
matical solution will ba madr explicit. 

2.13.2 Risk and Uncertainty 

It is rare for a decision not to include some degree of risk and uncertainty. 
In many cases, the risks can be identified before the decision is mads, and their 
effects can be included In the analysis. Some degree of control is thus obtained 
over risks and uncertainties, making it possible, for Instance, to specify how 
much risk can be tolerated. 
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L distinction should be mode bit«HD risks snd uncertainties. A probability 
can be assigned to any event that is considered a risk, but no probability can 
ba assigned to an uncertainty. An auntie of a risk is a saAbier making a bet 
that be will ires a rad ball from an tarn containing 5 rad balls and 10 whits 
balls, tbs possible putcoses are known, and the probabilities are 1/3 for a 
rad ball sad 8/3 for a whits bell. An example of uncertainty is making the 
sane bat where the number of rad bells and the number of white belle are unknown. 
In this cate, ell that can be said About the outcome is that a red ball or s 
state ball will be drawn. 

Xn practice, the distinction is not always dear. It may be known, for 
axaap'.c, that the maker at red belle is between five and ten and the number of 
whits bells equal ten. 8inee analyses under conditions at risk are preferred 
to those uuder conditions of uncertainty, sou effort mat ba made to learn mors 
about the eyatea and thareby reduce the amount of uncertainty in the decision. 

2.13.3 Optimisation Techniques 

The technique for optimisation essentially involves the application of 
effectiveness and cost oodels to all fcaaibia dasigne and selection of the de- 
sign which, according to tb'j criterion, is optimum. 

While this approach is conceptually simple, its implementation la virtually 
impossible, except for the moat simple p rob lasts, consider a problem involving 
fifteen variables, each of which may taka one of only two possible values. Nora 
then 32,000 possible system designs would have to be considered, a magnitude 
that would tax even the largest of the available computers. 

Techniques are therefore needed to reduce the amount of aatheamtics and 
confutation to a aisa reasonable for computer, geometrical, or seen hand solu- 
tion. In a sense, these techniques are sophisticated trlal-acd-error routines. 
Some of the more casmoniy used techniques, or fields from which such techniques 
are derived, are Hated In Table 2-3. The list la by no naans complete. A 
brief description of several of these technique* is contained in section 3 of 
the Uuid ebook. 
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I. 

Mathsawtlchl techulqume 


Birth and des*-h s^oo eases 

Calculus of finite differences 

Calculus of variation* 

Oradxant ci eory 

Buaerlcal amrootlamtlon nsthods 

Symbolic logic 

Theory of linear integrals 

Theory of no vi nun and mint— 

11. 

Statistical Technique* 


Baytsian analysis 

Decision theory 

Experimental design 

Information theory 

Method of steepest ascent 

Stochastic proceaase 

m. 

Programing Techniques 


Dynamic programming 

Linear programming 

Bonlinear programming 

IV. 

Other Operations Beoaarch Technique* 


naming theory 

Monte Carlo techniques 

Queuing theory 

Benewal theory 

Search theory 

Signal flew graphs 

Simulation 

Talus theory 


2 . 13 . ** JSSSSStS. MSSSii 

During the analytic of system coat or cost-effect iveneae, a g -at deal of 
emphasis Is nacasaarily placed on the three baalc typea of coat: research and 
development, inveetment, and operating. However, coate and benefits In another 
category are often overlooked during theaa analyses, they ere overlooked be- 
cause they do not Increase or diminish the total coat and effectiveness of the 
system being analyzed. These costs and benefits are called leverage effects 
in that they cone Into play when an alternative associated with the system 
being analyzed influences (acts as a lever on) the cost or sobm other charac- 
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teristios of another ifita. Leverage effects include those allied factor* or 
rtiwrti itUh in asternal to the system being studied Out ruqr bar* o signifi- 
cant tf not ovenbelmiag iapeet on tha final choice among system alternatives. 

lannp affeeta and not to «qf(MHd m a quantity of dollar*, tint, or 
ether unit*. Buy can bo aleply foot* or elreunrtaacee that ahould bo anty. 
•idorod in tbo decision. in tamo of too definition of c oat-off octivanaoa 
analysis given oar liar, a leverage affect i* presented aa one ccmp o nent of thv 
array of eharaatcrlatlea nantionod therein. 

Leverage offoeta can bo illuatratad by a simple nopli. A military agency 
ia to aolact for davel op— nt on* of thro* alternative powerplanta, a, B, and C, 
for uaa it a new helicopter, it io aaai **ed that the total system coat and 
effectiveness have bean aatiaatad for aach altarsatlva aa follows : 


Total System Coat 

Affsctivsneoa 

It la apparent that A la tha boat choice a Inc# it is the least costly and 
provide* the mm affactivanasa. O ow a v s r, Powerplant C nay have certain quali- 
ties '.ft*.* would permit its aff active uaa in a new tank that ia being developed. 
It ia east mil that the total coat of a new powerplant developed for the tank 
alone would also be 180,000,000, but that if powerplant C war* selected «d 
deve lo ped for the helicopter, its total coot would bo reduced to $50,000,000 
because of shared development coats. This savins of $30,000,000 in the develop- 
ment of the tank now naked C appe ar to be the boat buy. Ths $30,000,000 la a 
leverage effect, since the powerplant for the helicopter will still cost 
$80,000,000, but its development will affect a $30,000,000 saving In the allied 
tank program. 

It might be argued that la .arege affects could b# included in either the 
coat or affactivanasa values of tha system. H owever, the analyst is forced to 
isolate the problem and define a system associated with it in order to perform 
the analysis at a manageable level. Although the leverage effects are known to 
influence the decision, many factors mat be excluded if the eystam la to be 
represented by e model tad pertinent information la to be extracted from that 
model. Then, after tha modal ia applied to the various alternatives, acme of 
tha excluded factors, e.g., leverage affects, are reconsidered for the final 
decision. 

2.13.5 feterpretatlon 

Aa indicated previously, e modal of a cosplex process is usually incowplet* 
because of uneertaiatlee, eon-quantitative rector*, Inadequate date, and inade- 
quate consideration of the effects of the process on systems and operations at 


romimt a 

$ 60 , 000,000 

0.95 


tomim* B 

$70,000,000 

0.95 


Powenalsnt C 
$80,000,000 
0.95 




higher sehslons. xn ii^i omm the nwiti at tte cptlaUadteh p f o s os» eon 
lad*cat# tlw boot u«t« «UUd ths e Uplift cations, uinptln , mlrieUm, 
and amissions required to elreuwdnt U» volte. 

ft* effects of t h in e treowte -aw oust then M evaluated throng mm 
type ef modal f — tenoh ffo otew which, on ths tools of ths estates* results, 
nay mwl som critical daflelanriss thofe am Vo roetlfioO. 

BMmr, atm tbs aoot aodern aathemitlcal teahnfcgf oat o co urt sri will 
yield only partial analytic solutions, mainly boo suss at tho uaeortolntloo. 
fboso unsortslntlss often otfrt la tho ovoroll objective and, Mmo breodsr oon- 
teuts in tolog considered, It nay te neccseary to srantoo alternative objectives* 
Vs thns hurt tho enlarged problem of tint selecting tho options objective on* 
tho otooeteteO cftlan sot of constraints. 

lbs optlaisstlon process, therefor*, provides tho fr onsuo rtt for o fteol Is- 
Olsten. If ths process is besot on o correct fonrlstlon of tho problem ant 
application of s rsssonsbte nodal, the daclalm eon be critically evolaotet ant 
suitably aodlflsd. Me m , beeosss of our present inability to oplcgr o strict* 
ly analytical approach, the ssperleace and judgment of nsnsjcnsnt Inherit reopen- 
slblty for the final choice. 

*•^.6 Cmclusten 

Ideally, the criteria for o decision should be explicitly stated so that 
there is no doubt ccneomias tho acceptability and acc ur acy of e decision, b 
aost esses, however, explicitness la laposslbls tscusss of tbs anesrtetottss that 
prevail and bee sues of limitations of aval labia methods, for Instance, e state* 
meat about the reliability of o device la mean Incises unlsss o standard of moos* 
ureaOMt is given. Sven if this eta ter* is given. ths » tat Mint would still be 
nanlngicM uulees the methods for usblnc ths usssursnmt were available. Oh- 
certelnV.es about the validity of tho standard of measurement and the accuracy 
of tho method, and about whether the criteria ere proper In the first place, 
further ccupllcut* the establishment of decision criteria. 

ths appropriateness of decision criteria for military a yst sna is a contro- 
versial point, tho Weber of offensive weapons for a fixed cast, the number of 
targets destroyed for a fixed coat, the number at lives saved for o fixed seat — 
which should be uaedf Selecting e particular criterion for lack of o better one 
can create serious problems if it is tbs wrong one. H ow ever , nuking decisi o ns 
in ths sbssnee of criteria can alee have haiWul c onse gua na —. In understanding 
vj the complex tty of criteria is essential. 

the normal criteria are those which result in navlwn effect iveness for e 
given budget or e specified effectiveness at minima cast. However , the absolute 
value of gate or cost must not be overlooked, as if jouIS be in step If nsrlntilng 
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the tcUo of offoetlvMMM to cost. the ratio of effectiveness to coot la not 
••rurally an adequate criterion for making a choice among ccmpating ayatama. 

Tbs absent* of a standard eritarion does not preclude the analyaia of coat* 
aff act 1 v*naaa. it mom that at math Information at poaaibla on the tyatam nuat 
be derived for consideration by the decision makers. Although tha information 
cannot bn " wra pp ed op" into a single valid criterion, it can bo display ad in a 
nconor that facilitataa itt uaa in ooujunetion with tha daeielon-aekar's expert 
Jndpwnt* Thia requirement, however, placaa an additional burden on tha ana* 
lyat, sine* ha mat maintain a gr an t daal of flexibility in hit modeling to make 
bit analyaia adaptable to counting infotnatior requirement*. 

Md mum immit m mmm mm— no* 

lha major function of tha evaluation and faydback information process it 
to provide eonatant updating of previous input# and analyaia by usii* tha in- 
formation gained from tha atudy proeaaa aa it baeanaa available. 

for axanpla* it may baccma apparent after exercising tha effectiveness 
nodal that tha alternatives considered in the analyaia are all extremely vul- 
nerable to enany action. At this point la tha aystesm scaly* is/coat-ai f active - 
naaa proeaaa, tha analyst should re-examine tha mission profiles, threats, and 
hardumre characteristic* to determine which factors are contributing to tha high 
vulnerability. It the mission profile* are causing tha high vulnerability to 
scam avoidable tactic, this information should be fad back, to the decision- 
making level if nac votary, *o that tha ala-ion profile* can b* checked ror 
poaaibla change*. If tha threat appear* to b* causing the high vulnerability, 
the solution may ba to go back and consider a new alternative system, cr possibly 
cbmnga the performance req u ireme n ts of the system. Other considerations to be 
included in the evaluation and feedback process are the following: 

' Insure tut all aesumptljne and subjective Judgments used In the analysis 
are identified. The major assumptions should be explicitly stated at 
the beginning of the study effort and, if feasible, examined at the 
decision -making level to determine if the assumptions are valid. 

* Insure that all the uncertainties that occur in the ana lye is are treated. 
The uncertainties of future threats, envi ronm ent*, and perform*.-*: • 
characteristics may have probabilities and confidence level* associated 
with them, wd these should be explicitly stated 

* Xxamlne the output at every stag* in the systems aaelysls/coet-effectlve- 
neee process to determine if the result appears to be correct, keeultc 
that are intuitively unexpected nay lead to a determination that tome 
factor in the analysis ms inadvertently omitted. 
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* Lae lad* par— trie triatmi >t of inaftlcM and variable* found to M 
sensitive. Jt tbe remits of a study are sensitive to tbs swjtlos 
that a spate* will oe in operation for sap fifteen pear*, go book end 
recalculate for one, too, and five pears on either side of fifteen. 

Ml WWW KMT* 

toe; the need for, end tbs absence of sap ami tsbls substitute for, tbs 
dec islcc -asking function it recognised, a tt en t ion eon be directed to tbe infer- 
nevion reguiroMBts of tbo decision acker. preparing appropriate infoanetion 
and presenting it to tbo doaiaton-nakor ere tbe funetiona of tbo oneipet, 

Sssentially, anelpois involves providing tbo boot poeelble eotiasts of tbo 
offoeto of selecting verioea courses of actlcn. the decision noksr nuat decide 
ehieb sot of effects be is «6st willing to accept, (fa met also, of course, 
Judge the validity of tbo eatlnatoo presented to bin.) 

One pitfall that met be avoided in tbe eyetoas snalysls/coet effectiveness 
process concerns tbe lanunt- of detail tbe analyst presents to tbo d; a Is ion —her 
d» woes tbe results of tbe analysis, tbe analyst oust exercise eatrem care 
to bo sure that tbe dacls&n-nUrar is eaere of any relevant factors that were 
not considered in tbe analysis or that nay have been obecur«'d by the data 
i faction process cor by the eealysla itself. In snort, since dots redaction end 
analysis involve eons decisions, tbe analyst mat be careful not to sake decisions 
that aore properly belong in tbo Jurisdiction of tbo decislon-oaker. 

Xt is onphaaised that tbs ayatana anelpaia/erot-effaetiveneaa process dost 
not r epre s e nt a decisions it is a process that concludes by presenting to tbo 
decision -ns ter, in a useful forest, inf amotion and data that are essential w 
hie asking a proper decision, me array eioecito shown in table 2-4 for each of 
three eoepeting system represent e set of data considered by the analyst to be 
iaportant to tbe deelslon-mke?, Sowever, If the analyst etteapts one additional 
step — developing a single cost-effect! venssa index from sc m or all of those 
data — he nay have node sene decisions that should have been left to nsnagemnt. 
If, for exoaple, the analyst decides that the iadnx is, in essence, effectiveness 
divided by the product of coot and tine, but that priaary-nlasian effectiveness 
is five tines as iaportant as the secondary-alas ion effectiveness (Condition i), 
erttea 9 is apparently the preferred systea. This would be indicated through 
the use of weighty fee teres that is,K^ and Kj represent the weighted * Impor- 
tance* of the prim end secondary missions . m the txenpie just given (Condition 
X) - 5*g. 
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KSEHi 

Xffs^tlrsasss 

flat to 

7*Ar 

£t> 

C-w 

Mnn 

M 0 

MtiU 

Xbrsat- 

Htt 

mm 



*fP* 

■ B 

ggj 

K^st tan 

w* 

(Condition 1) 

* ®j) 


A 

to. at 

Hex 

m 

0.90 

0.70 

2 

0.0898 

0.0670 

X 

0.7* 

6H 

9 * 

0.95 

0.06 

3 

0.0306 

0.0866 

C 

0.9b 

It* 

r.ifit. 

0.70 

0.90 

15 

0.0810 

0.0051 


Oost-XffaetlrsMss (C-X) U dsf 1 a*C for tltl* ocfi« bjr tha squat luo: 

c - * H - 

in , . *,) o * 

for Condition I, ^ 

for Condition u, Kj - Cg 

Tbsrsfors for condition I, tbs C-X of ajrstaat A Is calealatod ar follows: 

c-i - ^ * KgI * 

(^ ♦ Kj) C T 

- S(.9Q) + 1(.T0) 

{5 ♦ 1) (14.8) (2) 

• 5.2 
177 6 

• .0292 

And tbs C-X of aystsn A for Condition n is calculated as follows: 

c -,. ‘' V 1 * 

♦ X ^) C t 

• U -90? ♦ 1(.7 Q ) 

(1 ♦ 1 ) (It. 8 ) ( 2 ) 
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. 2 ±_ 

59.0 

- .0870 

Therefore, if tfa# analjret dee idea that prlaary aist'on trfwWwiin U 
fl*a tinea «m imp o rta nt es the seoaoderr-nlaslon iffteUnMN (Ccnilti^ t), 

■ i* the preferred syatae, So n-«w # if the analyst decides that ttM prlaar? 
and saconda.7 missions art aqaaljy iaportant (Condition U), r/ttn a mas 
ifpMrt to b« th* preferred qntn. 

A favorite nethod for presenting the result* of a cost-effeeUvsneea stcdy 
la * curve that la plotted nine coat aa tba ascissa and tfr jctivanoea aa tha 
ordinate. 

Certain conclusions can be dram canceralt* tha ah*** ol this type ot 
eurva. Ih general, to* elapse (a large gain la affaevlvaoaaa f<n a snail in- 
c*eaae la coat) ara tha neat desirable. Too aaanplaa .are shorn In pleura 2-li. 
aquipnsnt A coat* 13,500 for an aftaetlvaaaea of 95 percent, goeevar, ojr a 
oall increase in tha coat, Bquipaent A can to given an effectlvaoeea of 99 
percent. Conversely, for Bqulpneot B, unless there is *. aeed for ar effective- 
aeaa greater than 96 percent, 00 acre than $3,000 should be spent on this equip- 
■ant. 



mmi% v stiMMW mm 


AMC*7fe.li1 


Ths print function of ths syttsn analyst Is to provide ths dscisian-jaker 
with «s Mch jrganlsed, rslovsnt lnfomstlon as post ibis. Severer, this Infers:, 
tlcn dons not sutaasticai ly identify ths preferred alternative, because • co-eon 
ralua-r lesur* camc-t always t>« developed. In Marine such final sslec.ion, ths 
decision -waiter Bust also account for tho limitations In ths quantified analysis, 
such as data laadequeciKs, and. line atrapticoi, and unrirtainty. Ths final 
select 1 <ju, ths refers, aunt bs based primarily or Ida decision -naher'a Judjnsnt, 
ths intonation in ths C-S array prowlf-** support for such Jude- sot. 
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There ere three broad classes of techniques that can be applied In ayatems- 
analysis and cost -effectiveness studies. The first class consists of techniques 
that are used to represent a system's behavior as a function of tlus — usually 
In a statistical sense. This description is generally probabilistic In font 
and thus relies heavily on the theories of probability and statistics. Moreover, 
sines the representations are often complex (both mathematically and physically), 
a computer la often required to manipulate or aolve the*. 

Within this first class, there are four techniques that are generally 
applicable In the eysteam-an&lysla/eoBt -effectiveness process. 

• Simulation • Sequencing 

• Queuing Theory • Inventory and Replacesont 

The second class of techniques applicable to systems analysis/cost- 
effectiveness is concerned with finding optimal solutions, l.e. , the maximisation 
or minimization of soste objective function within specified constraints. Within 
this class are the following techniques t 

• Linear and Ityraaic Programming • Analytic Models 

• dams Theory • Decision Theory 

• Information Theory 

The third class of techniques consists of those statistical and mathematical 
tools used by the analyst to identify relationships among such system parameters 
as coat, performance, ate., and determine how critical *he parameters are In 
the decision-making process. This dans Includes the following techniques t 

• Estimating relationships • Experience Curves 

• Confidence Intervals • Sensitivity Analysis 

Table 3-1 lists the various techniques introduced above, Indicates the 
general application of each one, and Identifies the section in this guidebook 
In which the technique is discussed > 
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S.1 StMUTIIR 

A simulation is a model (usually computer) that duplicates a system's behavior 
without actually employing the system. 

A simulation can be employe 1 in many t^pes of systems analysis. Some of 
the more important areas and circumstances are: 

• Environmental problems 

• Mathematical formulation 

• Lack of analytical solution technique 

• Experimental impossibility — e.g. , large-scale conflict 

. cost 
. Time 

• Training 
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gf mutations osn b* either analog or digital, and both hay* bom applied to 
a boat of problems. Within the sot of almulationa are several other ooncopta 
such as Most* Carlo, fleml ny graimn< Canoes, and Nodal Stapling. 

Analog simulations are nost often used as a am-uis of solving sets of 
differential equations or problems dealing with continuous functions. Generally, 
the &sfceas analyst is sore likely to encounter digital emulations in the 
exercise of his studies. 

The question night properly be asked: how is a digital simulation of a 
occplex systen obtained, say, for a forward-area air -defense problem? The 
following steps ire necessary: 

(1) The characteristics of the offense, defense, and environment are 
determined. 

(2) A general flow dlagrsa for the simulation is developed — for example, 
the flow between the threat, detention and tracking radars, end the 
intercepting missile. 

(3) retailed flow diagrams and submodels are developed — for example, 
the method of computing the lock-on probability for the t rooking 
radar. 

(4) Space and time coordination are developed throughout the simulation 
for each simulation element. 

(5) Statistical slses and constraints are determined. 

(6) Inputs ere incorporated. 

(7) The simulation node! is exorcised. 

An Important aspect of Nonte Carlo game simulations is the Design of 
Experiments for testing numerous variables and reducing output variance while 
reducing the required sample else. A formal branch of statistics is devoted 
to this problem. 

The applications of simulation techniques are manifold. They range from 
strategic or taotleal operations to management, to simply system operation. 

They provide a means by which the analyst can handle large numbers of variables, 
mathematically intractable relationships, and, most Important, uncertainties 
and alternative steps. 

1.2 mom mm 

Queuing problem* may develop whenever there are demands for service from 
a number of more or leas Independent sources. Queuing theory is a technique, 
based on probability theory, that supplies a means for mathematical analysis of 
this class of problems. 
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Infill of queuing (or waiting lino) situation* sro aossago flow in a 
oo— uniootiiono oenter, ousts— r servicing at * —pair faellity, — « flow of 
traffic through a bottleneck. Hup factors —at to con aids—d la the analysis 
of queuing problems. ions thoso a—*: 

• 9m probability distribution underlying arrival tlaaa 

• The probability distribution underlying aorwiolng tl—a 

• The nunbor of waiting line* 

• Tho n— I bar of servicing faollitisa 

• lh* qua— discipline 


With knowledge of than* faotors, tbs analyst oaa ofton p— diet such 
lnportant —suits as th* a— g* length of tbs railing lino and tbs a— gs 
Idle tl— for a service fsolllty during any specified tine Interval. 


The utility of tbl* method can be d— oast— feed by an example. Messages 
arrive at a oo— ualoatlons center on the average ones every 10 alnutee and 
with Poisson distribution: 


* T (n) - 


(») n e 

n! 


The service tl— a for r—oesslag the 


-w 


saagee are assu— d to be exponentially 


distributed p(t # ) 



with —ana of 3 nlnutea. 


The questions to be anawu— d a—: 

(1) Most la th* average nubber of — ssagts In the com unioatlons center? 

(2) What la the average length of th* queue that nay fora? 

(3) Assuming that —other neaa&ge clerk will be put on when a — aaaga 
would have to wait at least 3 nlnutea before being processed, 

what higher rate of arrivals can be tela— tad before another nan nuat 

be assigned? 


Par this particular type of a queue the following relationships can be 
derive d by —ana of queuing theory: 

• Average nuaber of — aaegee In the on— unioatlons center - ^ , 

wherj 

\ • average arrival rate • 0.1 per aln. 

U - eve— ge aervloe rate - 0.33 P«r min. 


Reproduced by permission from KAVAL OPERATIONS ASAI Y 8IS; Copyright 1968 by 
the l', S. Haval Institute, Annapolis, Md., p 238. 
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• Average length of "non-eapty* queues ■> -M- 

(4”X 

• Average waiting tlae - -4 n » 

the answers to tbo three questions posed, then* ere i 

(1) Average sober of — saages in the aystea - q.g^ O .l T 0»*3 —saages 

( 2 ) Average length of ntm-eepty queue - - 1.A3 neeaegee 

(3) Tolerable arrival rate *"- r -3-*A' - 0.16 aaeaagaa per alnute 

0.33(0.33-1 0 or 

10 Manages per boor 

To obtain sons Insight Into the underlying theory, consider the stagiest 
oese — that of the single -server queue with Poisson arrivals, Just discussed. 

The aasber of units In the eyetea is found by developing recursion relationship*, 
which axe governed by the previously sited factors.* 

Let n » the total nuaber in the eyetea (the nunber being aervloed plus the 
number In the queue), and T n • the probability of there being n units In the 
systea. Assum that the queue discipline Is such that an arrival novas lanedlately 
Into the aervloe area If the area la vacant. 

The probability of an arrival In a —all tine lner—tit, At, is X&t. 

The probability of e aervloed unit leaving In the interval t, t ♦ At 1st 

0 If no units are in the aystea at t 
liAt if there ere one or aore unite In the eyetea at t 

The probabilities of aore than one arrival or service, or both, ooourrlng 

9 

In the Interval are taken to be aero alnoe tiny ere proportional to At or higher. 

Consider the following two conditions 

(1) 0 units In eyetea at t + At 

(2) 0 unite et ti— t, no arrivals In At + 1 unit at t and 1 aervloe 
ooapleted in At 

These two events ere equivalent end thus their probabilities of occurrence 
nust be th« earn. 

Thus P 0 - P Q (1 - Mt) + P x fiAt, or P x • i P 0 


*K. Sasleni, et. al., Operations Baasaroh — Methods and Probleae . John Vllsy 
and Soria, Inc. 1959, pT IfeU. 
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Vo obtain L, nvijr tbs mm preostsr •< tbs m u lt lot 


*1 " *1 [ 1- < x + •*) At ] ♦ p o <***> 


This o*n tbsn bo 


*. - H* k* ■ i v*» ° * * 


and, bgr induotloc, tbls oan bo I t tw oo 


p n * $>" V**° 


HO WO TOT t 


oo that 


*a~ l 


'.-‘•I 


I*- 1 -! 


»»-(>- s> £>" 


Onoo tbo P n or* dotondnod, rolationohlpo ouob oo tboso usod la tbo prorlouo 
oxavlo con to found, it th> arrlrsl ond sorrloo dlotribvtlCT* booaao ooro 
ooo^lioatod, tbo nuBoor of waiting llnoo oed oorrloo faeilitioo I n o r ooo o o, or 
tbo quouo dloclpllno booowoo sort oonplox, ond tbo ooooolotod nathonstlos boeoons 
oorroopondlnpiy ooro dlffloult. 


Wbon tbo asthmatics boocnoo too oooploz for • alosod analytical solution 
or too costly, tbo opproooh omlayod lo Nonto Carlo. Vo lllustrato, consi d e r 
tbo oitOfMo oium of tvo ooquontlal oorrloo faoilltlos, ooob psrfondac tc taw 
known but net nsoosstrlly oiaplo distribution. Slailarly, tbo orrirals boro 
mm known but not nsoossarlly alaplo distribution. Par slapUolty, o«aln aosuao 
tbo oo as 'iuouo discipline os prorlouoly. Tbo doalrod onswor Is tbs expoctod 
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**■» hpent in tun qrttw. applying Mont* carlo techniques to the thro* 
distributions fcr aeon item In Aha system ant keeping trade of aba ra the itan 
la tbroagiBat vitas, tha tlM aaoh item qpenda In tba ayetam can ba deter- 
mined and, aqr ib>1a averaging, tba resulting expected tlas in tba aystwm oan 
ba aansrtglnad (subject, or eouroe, to statlatloal oonfldenoe requirements). 

Zn this manner, highly ooap a l loa tad systems with many patha and serving 
points, varied distributions, and queuing dieolpltne* can ba analysed. 

14 tfwm ui uhn moats 

j egu e oeln g la ralatad to tba ordar In which ur „ta requiring annloing ara 
serviced. It appUaa to a olaaa of problsau In which tba aaaaum of effectiveness 
la a function of tba aequenoe or ordar In util oh a nuabar of tasks ara performed. 
S n a anlm problems fall into two groups** (1) performing n tasks, aaoh of which 
requires prooaaalng on seas or all of a different machines; and (2) processing 
a list of n tasks with a naohlnaa, with tba daolaloa of tha n+1 task being wade 
at the completion of tba task. 

Unfortunate lx, both types of probleaa ara exceedingly difficult ; at present, 
aolutlora arc known only for wane treelsl cases of tha first type. 

A classical example of tba esqusnolng problem is tba traveling Salemsan* 
problem in which the salesman oust visit a series of locations, stopping at aaoh 
location only ones and returning to his starting point at tbs conclusion of his 
travels. An analogous operations problem la tba selection of messenger routes 
within a division. 

A further potential application la to use sequencing as a management tool 
la tba development of a com p l ex system requiring msoeroue tasks with various 
facilities or resources. Tha objectives era to determine tha optimum use of 
tha facilities through proper sequanoxng of tha tasks performed. 

To Illustrate the technique, consider a me saenger -routs problem In which 
five stops are to be mads and tha requirement la to find the route involving the 
minimus total distance to travel**. For this type of problem there ara (n-1) ! 
aubaata that must ba saarohad for a solution — in this case. A* o.' 2A. 

This problem could also ba viewed as an allocation problem, but complicated 
with tha added constraint that tha Messenger must not pass tha asms point twice. 


• Ibid, p. 250. 

**Xbid, p. 264. 
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IHOS^ 

A 

B 

0 

D 

8 

A 

• 

8 

5 

T 

1 

1 

8 

m 

3 

8 

8 

0 

5 

3 

m 

4 

7 

9 

7 

8 

4 

• 

5 

3 

1 

8 

7 

3 

m 


Uum ttt flltlMI Mtrlx OB tb* 
loft, *m tho wMm (t'o) 

point. (tetoi tho Ry • Ijj, tho q#t 

in pooorol imm this and not to oo. 
Also, tho ft^'o kno bom Mrtpwd 
Infinite nlsM to nom ttao fro* 
tho ppouoa.) 


He obtain o solution, firot nanlpwloto tho mtri* oo tel*. ot 



(8) Huh wiaooAgnad rooo. 

(3) Motk eeliana bovine soroo la aoricod row. 

(4) Norte row bovine ooo l g nw nto in norteod oolwna. 

(5) Aopoot (3) and (4). 

(6) ora* Uno throufh w o rto d rev*. 

(7) trow lino through wrtrod ooloono. 

(8) to loot onolloot umortod olomntj odd It to lntoroootloooj oobtroct 
fro« uawrfcod. 


tho onolloot voluo Is 1, and tho no* aotrlx 1* oo fcllowi 



3 

X 



5 


4 

B 
• 8 

8 » 
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Idont lfp mpm (tte >110— Man pc okl« tec no m boon solved). Cheek tte 
MpMMt A - B - I - A (Ml eolation). 


*7 the Best beet eolation bp a ip a nffin p iboat the anUleet noa-eero slanant, 
efclofa la 8. 



tte eeqpanee lsno«A-B-0<E-8 A, Jt* ooet of 8 tiles stew tto 
■Ini— iUmHUr, Bill la a «114 eolation. Da lenpth of the iwU la than 
8*S + t + 5 + l»15 nllaa. (Mata if the expansion hod been parfome c about 
the otter 2 alone nc. It mold have etrtlr mrm d the oeder.) me eolation 
la a aim— ruun. 


tte behavior of a ape ten that tee discrete states altb probabilistic 
trend tl mu ua| the atataa eon be repree e nt a d bp ebat la know as a terkor 
pr oeeae — a eo ndlt toaai pr oaa aa la uhleh the next transition depends on the 
abate the apetea is In and* for eone tppee of p coo era f on tte n presorting etatee 
nil. Xa tea sense, it la stellar to sepoanolnK* ■soever, tte pool here 
a deacrlba the nreten'e behavior atatiattaallp la taena of lta tranaltiona 
lta ability tele la each etate. 


Inventory sen be defined aa tte phpaiaal atook of goods kept on hand bp an 
organisation to pronote the efflelent nmia| of lta err el re. the eoete assoolated 
with ntlntalnlng aa inventory are nomally placed in *hree categories » 


(1) tte ooet or ordering goods 

( 8 ) tte o.itt or holding foods it Inventory 

(3) ft* onata of liwrorring shortages 


Tte prohlsn faoinp tte deelelon eater la tecfold: 


(1) Worn often should he replenish? 

(8) lev nueh should hs replenish? 

2fcv«a*ory -control tteerp la a eatresatloal spnroaoh to flndii* tte optimal 
re-ondar tins and quantity end le usually ba s e d on the values B»at v&Xl tea lades 
tte overall ooet of anintelnlat the law*. tor?.* 

^ tey rS Kg anSy •ffgnlsslqp free KA7AL OPfSATIOU AJUtfglAi Copyright 1969 by the 
V, 9* level Institute, Annapolis, **., p.&G. 
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The ayeten mb that ta thought of e# oonlotiag of outputs (Kwato foe 
tho good*} and inputs (roplaaiah— at of the poodo). 

Input and o u tp u t art bo Ascribed 1 b tonu of their respective rates 
a(t) and b(t), and the inventory level at any tint t is tbply 

U*) ” *<*>]«* 

'A* input and out pot rotoa do not assessor!!? '**vs to bo flaito bat thole 
tin integrals oast bo flaito. 

Cto inputs aad output* aro critical footers In solving tuvootosy probl— . 

X--t i n v e nt o ry oodols Include eas of tho throe typos of MhMpmna wpidlad 
these i 

$1) Continuous In tl— 

(9) Quorate and aouldlatant in tin* 

(3) Quorate and irregular In tins 

tho inventory preblen Is also oiasolflod In toms of tit* aamrt of knowledge 
regarding thaso factors; l.o* t C.a factors am either known and baooo datorvlaiotlo, 
or unknown ye hsaoo pa oSoblllotls 

harder, If tho par— at aro aro waataM with tins, tho problaw la said to 
bo static; if thay aro not, tho probl— la dyrosdo, with annreapoadlag lacrosse* 

In — thaw tlcal difficulty. 

The Inventory ayatsai Itself ssy U» eonplez, with various station (atrlee 
end parallel) end various levels. In addition, tho l i nks between stations 
oaa vary In fom {sli«ls, alternative, fusion), taken ea a Mule, the ayeten oaa 
be considered analogous to a network. 

A fine! consideration to be discussed concerning Inventory aodaia le ..** 
delay factor between re -order end replenia— nt. Tho principal difference that 
results fron including the delay is that further depletion cf inventory between 
the decision end the snivel of rep lace— nt* oust be considered. 

to illustrate the technique, eoreidet the aiapl* enable* of e single station 
that has a unit or* daswad rets bit) - > unit? per unit tine. Obits are re-ordered 
every T days end the re-order cost Is C. There if no appreciable delay In 
fllllrg orders, sc that the practice le to re-order vheaever the Inventory le 
cere. 


•Teeierd, gp. alt ., p. 71. 
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tha iMt «r iwmlon li wwii to ^o pr oportional to t ho nwbcr of wita 
hold and tho Iwgth of tlna tbay or* hoi*. ftau tho Inventory ooat hart 3a C^IT, 
liilf fl^ lo tho wit holding ooot par wit tin and X la tot aaowt In tha lnvan- 
ti C M P y . IV aaurt reorder a 1 i * ary T nut bo Wt, and tha inventory holding aoate 


\ m h f y* , (r'Mrt)dt - | OjBT 2 


total ooot par wit tint la 


ooot, dlfforaetlata rltto reepeot to f and equate to uni 




or 


T • 


< c£? *** 


the ftaUty to bo re-o r d e re d la than 

a - m - 



and tba alnim— ixn|( total ooat 1 * 



- M *s 

tig lac— ant theory la oonoaroe d vlth el Coat lone In ahioh ayet— performance 
teterloretaa aith tin* and tha ayetae oan i*e restored to its initial condition 
through ao— kind of action. ThJ probl— la to detemine —an thaaa action* 
ahe*14 ba taken. 
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3.5.1 Linear FregujUag 

Linear programing Is used to determine the values of a set of variables ia 
a linear equation that produce an extrema in the objective while the variables 
are subject to a set of linear constraints. 

Linear programming problems generally fall into two oates'-ries, assignment 
and transportation, although the latter is actually a generalisation of the former. 
Assignment problems generally deal with distribution between a number of alter- 
natives in such a manner as to maximize or minimize the total worth or objeotlve. 
Transportation problems generally deal with routing of units between a number of 
sources and receivers in c’ich a manner as to m ocimize or minimize the worth of the 
operation. 

However, the problems need not concern only assignment or transportation for 
lire ^-programming techniques to be applied. Any problem that can be formed as 
optimizi.ig a linear expression subject to linear constraints can be treated. 

A mathematical representation of the linear-programming problem is simply 


it 

\ax “ ^ a ri x n 


subject to 

* 0 and 
N 

Y b mn x n • d m ; 1 » ••»** 

nil 

There arc « number of variations in forming these relationships, such »a the 
direction of the ineouality and whether the purpose, is maximizing or minimizing 

A number of techniques have been developed for solving linear-programming 
problems. Two of these, one graphical md ore analytical, are treated below. 

Consider the problem of twe types of helicopters, A and B, and the following 
Circximatances: 

. Type A carries 30 troops; B carries 20 troops. 

. There are fifty pilots available. 

• Type A requires two pilots; B requires one pilot 

* There are 40 of the A-type helicopter end 20 of the B type, 


The objective is to move the maximum number of troopr. 
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Th* above ■ tat omenta can be changed into the following mathematical 
expression: 

Maximize 3QX A + 20Xg are the number of the tjrpee A end S used, 
subject to the following constraints: 

(1) 2X A + Xp * 50 

(2) X k e 40 

(3) *b * SO 

First consider the graphical solution shown below. 



The procedure followed to find the optimal solution Is as follows: 

« Plot inequalities - Equations 1, 2, and 3. 

. tfote region allowed by each - inside crosshatuhed lines. 

. Note solution region. 

• Plot objective function. 

• Move objective function (parallel to Itself) away from the origin. 

. Note maximum distance point (lost point in the solution region that the 
objective function touches). 

The solution here is to use 20 of B, *5 of A. 

It should be noted that constraint number 2 could »ave been neglected with- 
out changing the solution in this example. 

The graphical method is a quick and easy method for solving linear-program- 
ming problems, provided there are only two variables. For three or more variables, 
analytical techniques are required because the solution space Is no longer two- 
dimensional. 
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The an* lytic technique described her* 1* th* simplex technique , «w theory 
behind it la complicated, but th* application la relatively slapla, although 
tedioua. (The technique la readily programmed for solution by ocoputers. ) 

To solve the sample probata, it la flrat neoeaaary to write the ayatea of 
inequalities ( constraint equation*) aa equalities, by introducing a aet of slack 
variable* — Sg, and S^: 


2X A + *B + B l - 50 
X A + Sg - 40 
Xg + % - 20 

Then, rewrite the objective function aa 

-3CX a - 20Xg + - 0 

where M represents the term to be maximized. 

Now construct a matrix of the coefficients of Equation* 3. through 4: 


( 1 ) 

( 2 ) 

(3) 

(4) 


2 

1 

0 

-30 


H 

i 

0 

1 

-20 


1 

0 

0 

0 


S 2 


0 

1 

0 

0 


i 

0 

0 

1 


K » 

0 50 

0 40 

0 20 


Objective Row 


t 

Objective 

Column 


Designate the M column as the objective column and the last row (objective 
function) as the objective row. A feasible solution is present when at least 
two of the columns, other than the M and N columns, contain exactly one 1 and 
all the other entries are zeroes and all the l’a are not in the same rows. 


For the matrix shown, there is a feasible solution: 


- 20, thus making X A , Xg, 
i.e., no troops are moved. 


S 1 - 50, 


40, 


M - 0. However, this is not the optimum solution; 


To check whether the solution is optimum, examine the objective row to aaa 
if there are any negative entries. If there are no negative en cies, the solution 
is the optimum one. In this case, there is a -30 and t -20} thus the solution Is 
not optimum, and the following procedure is carried out: 

(1) Determine the most negative element in the objective row and identify 
its column (the X A column in this problem). 
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(2) Divide ea«h positive • learnt in the selected column Into Its 
corrv spending ro> value In the K column. 

(3) Circle the element producing the amallrat ratio (the element 2 
in the column, which has a ratio of 25). This is known as 
the "pivot" manner. 

(4) Next normalise the pivot number and make all other entries in 
tho pivot column aero. This la donv by first dividing every 
element in the pivot row by the pivot number to obtain a new, 
normalized pxvot row. Second, fo r each other row, multiply the 
normalized pivot row by the negative of the corresponding pivot - 
column element, and add the two rows to obtain a new row having 
e zero in the corresnonding position in the pivot column. 

For this problem, the normalizing is accomplished by dividing the pivot row 
by two, then multiplying the new pivot row by -1 and adding element by element, 
to row 2 to obtain a new row 2 . Row 3 is already 0 in the pivot column, there- 
fore, nothing h«s to be done to it. Finally, multiply the normalized row by 30 
and add it to row 4. The resulting matrix is as follows: 


h 

h 

fl 

h 

5 . 

M 

N 

1 

* 

i 

0 

0 

0 

25 

0 

-* 

-i 

1 

0 

0 

15 

0 

1 

0 

0 

1 

0 

20 

0 

-5 

15 

0 

0 

1 

750 


This procedure is repeated until there are no longer negative entries in 
the objective c jui, and the resulting solution is optimal. 


The next pivot element s row 3, Xg column. The resultant matrix is S3 
follows : 


■v 

3 

H 

% 

k V - 

% 

M 

N 

i 

0 

i 

0 

-* 

0 

15 

0 

0 

-± 

1 

-i 

0 

25 

0 

1 

0 

0 

i 

0 

20 

0 

0 

15 

0 

5 

1 

650 

Since there is no longer a 

negative 

element in 

the objective 

solution is optimal and equal to X A » 15 , 

*8 

■ 20, 

and 

M - 850. 


A necessary condition for the formulation of linear programming problems 
is a linear set of objective functions and constraints. However, there are many 
situations in systems analysis — when one or more of the functions are expressed 
as a product equation in the variables — in which this technique can be applied 
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but not *11 tfci aquation* are linear . This often oeours whan kill probabilifelee 
of target* art balng da t trained. In suoh a oast, the aquation i* linearized by 
converting to the logarithms of th* function and optimising on th* log (which is 
monotonia to its antilog), 

To illustrate, consider th* ca*e in which there are two type* of weapon* and 
three type* of target*, with being the kill probability of th* target 
type by the 1 th weapon type. The objective i* to deteralnc the allocation of 
weapons to target* to maximize kill probability for at least on* target. This 
is th* same a* minimising the probability of not killing any target, let P denote 
this probability. Thu* 

p - (i-P n ) m (i-p 12 ) N12 U-p 13 ) h13 (i-Pgi ) 1 * 21 (Wb)“ U-p 23 ) HS3 

Taking the logarithlm results in 


i-2 i-2 

>3 & 

N lJ Lo« (l- p ij) 

i-1 i-1 

J-i J-l 


This can be minimized by maximizing Log - -Log P. Thu* the objective 
function is to maximize 

i-2 

J-3 

log P - £ 

i-1 
J— 1 

subject to the constraints cited. 

3.5.2 Dynamic Programming 

In dynamic programming, there are no restrictions on the set of equations, 
nor are there any general algorithms for problem solution. Dynamic programming 
wee developed as a means of studying decision processes and determining the 
sequence of cessions that results In optimizing a predetermined objective func- 
tion. 


In defining this sequence, Be 11mm* (who is the originator of this method) 
set forth a principle of optimality stating that an optimal policy was one which 
insured that each decision, in the sequence of decisions **ns the ontimum decision 
with respect to the conditions resulting from the prior de talons. 

Sc ie recent applications of dynamic-programming techniques Include: 

. Determining thrust-control policies and fuel consumption regimes for 
putting satellites into specified orbit altitudes with maximum horizontal 
components of velocity 

•Bellman, R., Dynamic Programming, Princeton University Press, Princeton. W.J., 
1557. 
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• Determining optimum sttging ratio for missiles (how May booster stages of 
what elms remit in moat efficient missiles) 

• Establishing optimum inventory control schemes for interacting inventories 
at different locations 

m suuuury, dynamic programing selects the optimum sequence of decisions to 
establish a polloy that will bring a maximum return. 

1.1 UK TNCMV 

dame theory is a mathematical theory cf decision-making by contestants with 
various strategies. Originally , It was developed to handle business and economic 
problems; however, It has found extensive application In military systems and 
operations analysis. 

The theory la defined as a mathematical demonstration that if opposing 
Interests aot rationally to achieve desired ends that can be set forth validly 
in a numerical scale of expected returns, returns that vary according to the 
success of various plans, the appropriate strategy for each aide can be deduced 
mathematically.* 

The following terms are used In discussing game theory: 

• Game — the set of rules that define what can or cannot be done, the 
size of the beta or penalties, and the payoff methods 

• Play of the Game — one complete run through the game. Including payoffs 

• Zero-Sum Game — a game In which the gains of one side equal the losses 
on the other 

• Strategy — a plan of action that is complete and ready to use before the 
game commences 

• Person - one of the opposing Interests 

• N- Sided Game — N opposing persons 

. Pure Strategy — a decision always to follow a particular course of 
action 

• Mixed Strategy — a daoision to choose a course of action for each play 
In acoordanoe with some probability distribution 

• Value — the -^pectcd gain in one play of the game with all playera 
uaing atable optimum atrategiea 

A competitive game has severs* characteristics worth noting**: 

• There is a finite number of persona. 

• Eaoh person hes a finite set of strategies. 

•Seprodueef by permission from NAVAL OPERATIONS ANALYSIS; Copyright 1968 by tne 
U. S. Navel Institute, Annapolis, Md., p.30. 

**S«slcni, op.cit. , p.156. 
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. Strategy choices art ®*da simultaneously. 

• There is an outooae of a play that determines r aet of payoff* to eaoh 
player. 

The amplest gene la the two-person/iaro- am game. This game la Illustrated 
by a problm In which the oo— anger of a wilt 1* planning to employ a o o a a w nl- 
oatlon system and he baa four candidate ny stems, while the enany ooaasander has 
five type* o£ jamming equipment to ejploy. The payoff for eaoh combination of 
cosMEunicatlon syatem/Janmer la measured In terua of the expected error probability. 
The problem is to select the etretegy to be employed by each omander. Aaaume 
that through analysis the following payoff matrix was determined. 


Jamming 

^v^^Syateo 

Communication \. 
System 

I 

11 

1 

III 

IV 

V 

Maximum 

Error 

A 

0.1 

0.7 

0.8 

0.6 

0.4 

0.8(111) 

B 

0.4 

0.5 

0.6 

0.3 

0.3 

Q(m) 

C 

0.8 

0.6 

0.7 

0.7 

0.1 

0.8(1) 

D 

0.4 

0.4 

0.8 

0.8 

0.2 

0.8(111) 

Minimum Error 

0.1(A) 

0.4(D) 

64(B) 

0.3(B) 

0.1(C) 



The commander's objective is to select the strategy that gives him the 
minimum error probability, while the opposition desires to choose the strategy 
that maximize-s the error probability. 

The approach taken is to examine each communications strategy and detsrmine 
which results in the poorest return, thus reflecting the poorest expected return. 
This information la shown to the right of the matrix.. Similarly, eaoh Jamming 
strategy la examined for its worst oase, and the values are shown below the matrix. 

Each comb* ndei then selects the best of his worst solutions as a strategy 
(circles appropriate values). Thus, in the rows, look for a Min-Hex solution and 
in the columns, a Max -Min solution. 

From the matrix observe that communications system B and Jrawer .rvttern III 
would *e chosen. 

Note that in this caae both strategies ere defined by the seme element. 

Such a solution is known as a Saddle Point, and the resulting strategies as Pure 
Stretegiea. The value of the game is 0.6, and if either side uses e different 
strategy, his expected return will be reduoed (in this osse the error probability 
would Increase to the communicator or decrease to the Jammer). 

If no Saddle Point occur*, the bet* strategies are mixed strategies, and 
the game (tuition is the set that maximizes the expected return. To illustrate 
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this, consider the iu< problem as above, but with the (B, ZXZ) element onanged 
to wy, 0.1. n>e new matrix la as follows: 


Jamming 

N. Systam 

Ccnsunlcat ioJi ... 
System \ 

I 

II 

HI 

IV 

V 

Maximum 

Error 

A 

0.1 

0.7 

0.8 

0.6 

0.4 

0.8(111) 

3 

0.4 

0.5 

0.1 

0.3 

0.2 


0 

0.8 

0,6 

0.7 

0.7 

0.1 

0.8(1) 

D 

0.4 

0.4 

0.8 

0.8 

O.i: 

0 . 6 ( 111 , rr) 

Minimum Error 

0.1(A) 

@0) 

0.1(B) 

0.3(B) 

0.1'C) 



From the matrix observe that system B represents the optimum strategy and 
Jammer II represents the best counter- strategy. However, there is no Saddle 
Point; hence, pure strategies no longer exist. 

The first step to the solution of this problem is to try to reduce the 
dimension silty of the game. It can be seen that no strategy dominate e within 
the rows. However, within the columns, column IV dominates column V in every 
row. Hence, column V is dropped from further consideration. Of the remaining 
matrix, row B is now seen to dominate rou 0. Carrying the elimination procedure 
to its limit results in the following: 


II IV 


0.5 

0.3 

0.4 

0.8 


Now let X - (Xg, Xp) and Y - (Yjj, Yjy) equal the optimum mixed strategies 
of the two sides. 

The gain la now a random variable, g, and the expected value to eaoh side 
is 


E(g; x, y) - £ a tJ x t yj 

1J 

E(-g; x, y) - 

1J 

bet Vj ar.d Vj, represent the expectation of each side, which is to be 
optimum. 
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Thu* itittHlu amt be chosen so th*ti 

«(«,. V ¥) s Vj given X Q 1* optimal oomBunloatlon system strategy 

*(-*) X* X 0 ) * V 2 given X Q 1* optimum jrsmer system strategy 

»no# the gw* is sero-ma, V x - -V 2 and X(*j X Q , T 0 ) - V, which aeens that 
If both sides us* their optimal Kin-Max strategies, their aohleved gains oolnolde 
with their Kin-Max expected gains (or in this case their mxtmm) is being mlnl- 
mised. 

Thus 

*(ti * 0 , *) * V 

T(g i X, T ) * 7 
0 

Substituting for ths sxptattd payoffs yislds 

¥u(°. 5 Xb + 0.4Xp) + + 0.8lp) t V 

XjjtO.SX^j + O/iXjy) + XjjfO.ttjj + 0.8r w ) s V 

In addition, 

*B * r T> * 1 
Ijj + * 1, and 

X 1 and Y t * 0 


The above Inequalities Imply the following relationships; 

0.5X3 + O.JfXjj * V 

0.3X3 ♦ o. 8 x d * V 

0.5V XI + 0.37^ * V 
0 A'ljy + o.aijy * V 


Thus there are five unknowns and ten relationships. Not all elements can 
be *#ro, but It Is possible for equalities to hold In the four Inequalities. 

Thus 

0.5X3 + O.^Xp - V 

0.3X3 + o.dXj, - v 


O.SYjj ♦ 0.3YJY - v 

o.tijj ♦ o.exjy - / 
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S + 1 


Solving thi* 9it» leads to 

S/3 
* 4 - V3 
I n -V« 

% ■ V* 

Thu* by using ocasuni actions nt B two-third* of the time end art D the remainder, 
the oommander will realise a payoff of 0.457* where** if he were to have stayed 
with hi a bast pur* strategy, ha would have been sure of no bettor than 0.5. 
Similarly, by Jem tins with Jaaaer number XI flva-alxtha of the time and jammer I 9 
the renal ruler, the enemy la assured of a payoff of 0.467* where** by following hi* 
pure strategy* he would not have bean aura of doing better than 0.4, 

Oases with large aatrloe* are often tadioua to solve by use of the technique 
just described. However* it is relatively simple to obtain an approximation of 
the exeat solution* 


Consider the reduced matrix of the sample problem: 
II IV 


B 

0 


0.5 

0.4 


0.3 

0.8 


i.i 

1.9 

2.7 

5.5 

4.3 


0.3 


0.8 1.3 1.8 2.3 


© © © © © 



- 2/3 

-1/3 


» JU. 

I * 
3/6 1/6 


• Seale.it, £g. £it., p 170. 
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Solution rules are •• follow < 

(1) So loot tho row an<! plaoe under u'rU (0.5* 0.3). 

(2) Circle a** lie at value la tho row and writ# oor responding ooluan 
to right of tho matrix (0.3. 0.8). 

(3) Circle largo at value In ooluan and write oorre spending rot and add 
to laat row* 

(4) Circle oaalloat value In row and write oorrespondihf voVjn and add 
to last ooluan* 

(5) Repost process N times. 

The approximate strategies after M iteration! are then the nuaber of olroled 
values ut^lded by N for each choice. 


Solving this ay a tea lead* to the following values: 


3^ - 0,667 
*0 - 0.333 
? u - 0.723 
? IV - 0.277 

Note that the values are of the right o-Jer# 
particularly rapid. 


Xg - 0.667 
Xg - 0.333 
Y h - 0.833 

! r. * 

but their convergence la not 


The upper and lower bounds for the gwme can be determined by dividing the 
highest number In the last column (8.4 after 18) by the nuaber of Iteration.!, and 
by dividing the lowest number In the lart row (8.3 after 18) by the nuaber of 
Iteration#, Thua, for this example, the value i* 


o.«5 a V • 0.468 


while the predetermined answer was V - 0.467, 


Aa the number of aides, the number of moves per play, and the dimension* 11 iy 
Increase, the ecmplexlty of the game solution increase# correspondingly. 

it TKHt 

Information theory ia * relatively new tool vo the systems analyst. Its 
lnitla, and scat frequent applications have, of course, been in co wun l cation 
system probie-*. However, U ha* received other application in such divers# areas 
a* si8siie-guiiar.ee wnd salr.tenar.ce-repor? ing-systea anaiyaea. 


ty .h x*si»g the opposite value f rtxi preceding ’Lme . 
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Zb gatw *1, th* theory oan be applied to any situation in which there is 
ur*oertalB«y to be reduced or, acre particularly, « here there le e aouree that 
oaa i»eerteiat t connected to an information alnk by a channel that nay perturb 
the aouree o ut p u t#. 

Tine "information*, aa ueeo here, doe a not repreeent • body or data, but 
aerely the aacunt of uncertainty that hae been reduced. 

A poesible application of Information theory ia to Mature effectlveneaa 
for a oanmnloatlon ayeteu, i.a., hoe auoh Information la conveyed by the ayatea, 
or hoe feat it la ooaveyed. 

for a quantitative treatment of this subject, aeveral concept# oust be 
defir- ■: , in rat, .information le defines as being exp**es»ed In K-aivy units, given 
by I - - Log^ t , -here P, la the probability of hiving selected nesaage 1 from e 
aouree containing a symbols. The most frequently used oa»» ? a tne binary, in 
enloh there are tee aynbola — (1, 0). Thus for a neaaaga consisting of e single 
symbol, the information is erprta**d in binary unit a and written aa 

1 - Lo*^, Pi 

for ample, the Information oonveyed by flipping e lagltlc .t« coin la 

X - -teCgty) - 1 bit, 

while the Information conveyed by e t,«o~h„*ded coin la 

I <- -lioggCl) « 0 bit*, 
l.a., the outcome le known In advance. 

An important concept In the study of infomatlon fa that ef entropy; ai»ply 
stated, tMe la the average uncertainty © f a source or message . Thui 

"•Ivi* $>,«<**. 

i l 

To illustrate, consider the eriropy e-r*.f ' -rued lr. * two-digit neeas*# {V-inary} 
having th* following sea#agt-popuia*ic«: distribution: 

StHSMS. frobaoHltj Sendi ng 

U 1/V 

oi i/C 

10 
00 


1/3 
V * 
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Thus 

H - ^Logg 2 + 1/8 Log 2 8 + 1/8 Log-8 + $ Logg 4 - 1 3/4 
bits of Information on the average Is transmitted by a message from this souroe. 

It can be proved that the maximum entropy Is achieved when all of the 
messagee are equally likely. Further, when the s'Tubola are Independent with 
the same distributions, the entropy of a message of length n Is n times the 
entropy of a single symbol. 

The next concept to be defined Is that of a channel. A channel Is described 
by an input alphabet A and an output alphabet B and a set of conditional prob- 
ability es [PlBjlA^)] , termed the channel matrix, that are the probabilities 
of receiving message Bj given was sent. 

To illustrate, consider a simple binary source having a symmetric channel — 
that is, the probability of an error's being Introduced on a one is the same as 
the probability of its being introduced on a zero. Symbolically this is expressed 
as follows: 

A B where the channel matrix is: 



Associated with the notion of a channel are several other quantities worth 
mentioning s 

« The a priori entropy of A - K(A) - 

• Th ® Posteriori entropy of A ■ H(Ajbj) ■ - ^ F ( a l b j) "°8 PfAjb^). 

r , A 

• Conditional entropy H(A|B) - \ H(A|b,) P(bj), which is called the 

equi vacation , B 

• Mutual Information (the information provided by the obr rvation of an 
output symbol) - I (A;B) - H(A) - H(A|B) 

H{A,B) 



■? 


?{a) Log P(a). 
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Some Interesting properties of mutual information are the following: 

• It is always non-negative, i.e., I(AjB) a 0, 

• If channels are cascaded, they will tend to leak information, i.e., 

I(A;3) a I(A:C). 

• Mutual information is additive i.e., I(A;B,C) ■* l(A;C) + I(A;B|c). 

Channel capacity is a measure of the ability of the channel to transmit 
information; it is defined mathematically as C - Max I(A,B). 

PUj) 

Channel capacity is commonly expressed in information unite per unit of 
time. The notion of channel capacity leads to one of the fundamental theorems 
of information theory: "If the average amount of information per message from 
a source is H and the channel has a capacity of C, then it is possible to encode 
the messages so that tney may be transmitted over the channel at a rate R which 
has a maximum value of 3/H."* 

If the concept of a noisy channel is now introduced, the preceding can be 
modified to "if the rate of transmission is less than the channel capacity, it 
is possible to encode a message for transmission so that an arbitrarily small 
percentage of errors may be obtained."** 

The preceding discussion concerns discrete messages. However, a completely 
analogous development exists for the continuous case, wherein summations are 
replaced by integrals and discrete probabilities by density functions. Thus the 
entropy of a continuous source would be given by*** 

H *= -J p(x) log p(x) dx 

Here entropy will not be unique but will depend on the co-ordinate system 
used to represent the variable. However, in the noisy-channel situation. It is 
the mutual information that is of Interest: 

I ,(x;y) «= H(y) - H(y x) 

> 

p(x)log p (y) dy - 

ThiB equation represents the difference in entropies — one term representing 
the received signal and the other term representing the effects of the noise. 

Then, as long as both terrs poeses3 the same units, the solution will be unique 
and hence not de-enoent on the co-ordinate system employed. 



-l 


PU) dx 


r. 


p x (y)iog 


p x (y)dy 


Presr. I960, p. 

' e *Ihid., p. 39? . 
* - '*Ibi-i.. r-606. 


al„ 

590 . 


Ope r atl.ns Research and Systems Engineering . 


Johns Hopkins 
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According tc Shannon's Thaorem, an amount of information par sample point 
can be sent over a noisy channel ae glvan by the maximum of the aquation above. 
However, the channel must be evaluated In terms of the specific ahannel used. 

The channel Is restricted by the bandwidth available and the power available for 
the signal waveform. For a signal of average power P An the presence of narrow- 
band Oauaeian noise of average power N and bandwidth W, the channel oapacity is 
given by: 

C - W Log (1 + j}) 

Thus the trade-off between power and bandwidth Is shown. 

When the basic relationships pf information theory discussed above ere 
applied to a systems-analysis problem, the components of the system are repre- 
sented as channels with appropriate characteristics and the inputs and outputs 
correspond to the information passed by the system. 

To demonstrate the application of these techniques, consider a slightly 
different example — a maintenance system A simple system experiences three 
types of failures and exhibits four types of symptoms. Analysis of symptom/failure 
frequency data yields the following matrix, where each element is the number of 
times the corresponding failure/symptom combination was experienced: 


-^. Symptom 
Failure ' 

S 

1 

S 

2 

s 

3 

-=* 

co 

Totals 

F 1 

5 

4 

1 

0 

10 

F 2 

2 

1 

2 

5 

10 

? 3 

1 

2 

5 

2 

10 

Totals 

8 

7 

8 

7 

30 


The first step in the analysis io to convert the elemental values to prob- 
abilities. 


-^Symptom 

Fa i lure 

S 1 

S 2 

h 

S 4 

Totals 

F 1 

1/6 

2/15 

1/30 

0 

1/3 

i 

F 2 

1/15 

1/30 

1/15 

1/6 

1/3 

? 3 

1/30 

1/15 

1/6 

1/15 

1/3 

Totals 

V 15 

7/30 

4/15 
— 

7/30 



1 
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The aver*gt information contained In a symptom la 

4 

H{3) - P(S 1 ) Logg P(S 1 ) 

1-1 

" ■ if ^*2 15 ' 15 ^2 35 

(3.907-2.000) (4.907-2.807) 

“ 1 ^( 1 . 907 )+^ ( 2 . 100 ) 

- 1.997 bit# per symptom 

Similarly, the entropy contained In the failure# 1# 


■l *<']> «»,> * j - 1.585 bits per failure 


The Joint entropy In a symptom 1# found directly from the symptom/f allure 
matrix (note that In this case the matrix Is not the channel matrix): 

H(S,9) • -V f P(S.,F.) Log* PfS^F.) 


1-1 J-l 

- Lo « 2 F “ h U ** h ’TO ^*2 33 " 1 ? Ug 2 !!? 

- * (2.585) + (2.907) + ^5 (4.907) + ^ Log 3.907 

- 3.233 bits 

The Information transmitted from symptom to failure (i.e., the mutual infor- 
mation) is given by 

I(S,F) - ti K o) + H(P) - H(S,F) 

(This can be" -derived from the earlier expression for I that Included equivocation.) 


Thus, 


I(S,F) - 1.997 + 1.585 - 3.21c - 0.369 bits 


One criterion that can be apjgiled is the efficiency of transmission, defined by 


US, Pi _ 0 t 


H(P; 17585 


0.233 or 23.3* 
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Other items that can be determined ere the equivocation, 

H(A|B) - H(;>,3) - H(B) - 1.628 bits, 


the channel matrix, and the capacity. 


The system can now be investigated to determine the value of troubleshooting 
strategies; the effects of regrouping of component s to reduce troubleshooting 
clmes; and. In the case of AIDS type systems, the effectiveness of the system. 


3.1 ANALYTIC M68US 

Strictly analytic methods are another means of optimizing that the system 
analyst can employ. One such technique that is common in calculuB is equating 
the derivative to zero. However, two other techniques are worthy of mention: 
Lar.chester's equations, and the calculus of variations. 

3.8.1 Lancheuter'a Equations 

Lancheeter'a equations deal with the Interactions of opposing sides in a 
dynamic battle. In their simplest fom, lanchester's equatione state that in a 
multiple engagement "the v rail effectiveness of a force equals the average 
effectiveness of the individual unite multiplied by the square of the number of 
"jiitc engaged. » In mathematical terms this means that 


dB 

■m 


-kjlt 


dB 

3* 


-kgB 


where B end R are the numbers of blue and red units, respectively, t is time, and 
^ and k 2 are unit effectiveness factors. 

This signifies tr.at on the average each unit will in a given time score « 
certain number of effective hits, thereby causing the number of units killed to 
be directly proportional to the numerical' strength of the opposing force. 

These equations have been subsequently modified to incorporate other factors 
affecting for'® •trrrgtfc, such s« (uvUuolion rates. 

For example, one such modification la to write the two equations as being 
expressive of 'national effectiveness in a whoje war.** These are written aa 
fo 1 ' ,’ws: 

§ - h * °l n r * 

H * P B - °L% - e R^R 

*3, Merrill, et, al., " Operation Rrasarch, Armament, launching ", Prlnclplee of 
duided Missile Design , P. ^an strand Company, Inc., 'pllT. 

,p. lit. 
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where P - production 

0 L - operational loss percentage 

e ■* effectiveness 


Number of enemy deetroyed 
Number of frlendlleo engaged 


N - number of forces engaged 
B,R ■ blue and red, respectively. 


The equations may be further complicated by introducing probabilities Into 
the picture and by exercising them through a simulation, using Monte- Carlo teen- 
nlques to determine engagement outcomes under more realistic con** ti.ns and with 
more variables Introduced. 


To illustrate the equations simply, consider the folic wing nur.crlcal 
examples. Sides A and B have 10 units each. If all unite have equal effective- 
ness per unit of time, the engagement will be a draw. Let the effectiveness 
(kills per unit of time) of A be 0.1 and B 0.2. 

A q - 10 
B Q » 10 


Thua, 


dA 


dB 


0.2B 

0.1A n 


The resulting time histories for the two sides are approximately as follcwss 



Time Vn.it u 


Thus in this simple example B wins, losing but three units while all ten of the 
opposing A are lost, and yet Its unit effectlvensss was only twice as good as 
A'a, If B's effectlveneso were raised to 0.4 while A's remained the same, A 
would lose all ten ag»j.n, but B would lose only 2. Furthermore, the engagement 
would require three time unite whereas before it required seven. 
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yrcs th* e*«rpj . tfts tr»de-nff • he*v t »r rvssber* and Mil*. effectiveness {and 
time) are apparent . Thu#, fey u*«r of these equation*, optimal strategies could be 
deviseii , 

3.8,2 Ca lculua of V a riations 

Tltt urlculus of variations is an analytic method for dealing with problems 
of mtucima and minima, Ir. particular, it deals with finding tha as tram* of Inte- 
gra > a of one or more unknown functions. Thus, In tha cslcv’us of *«ristions, the 
type of prcblea that la addresaed la aa follows: find F such that 

-t2 

/ F(x,y,*, X ", y s', t) dt * Maxima or Minimum 

J tl 

The mst hematics involved In solving a variational p rob la* ia quite complex; 
however, electrical or mechanical engineers have probably encountered problama 
that could te solve! by the calculus of variations or have applied results that 
were derived from ins application of tha calculua of variationa. For example, 
tha derivation <f the optimum filtar la a direc application. In thla caoe, a 
tn.;( signal, Y?t), and a received aignal, y(t), (at the filter output) are given, 
and It ia d* -fired to find the filter transfer functions such that 

v 2 

iy->) dt - minimum 

A aecjrj example of tha uaa of this technique ia tha derivation of the 
equations of motion of vibrating membranes, plates, etc., from an energy stand- 
point (e.g., urtng Hamilton's Principle). In this case the problem ia formulated 

** 

fU -| 

/ . lffr,v,a,t) - K(x,y,t,t) - A(x,y,z,t) dt - Minimum 

7 tl L J 

where 0, X, A, re, actively, represent the potential, kinetic, erd applied 
energies in the ay.:v»m. 

The ye to the solution of these proolems is to consider ths 

functions *i tneju- end points but free to vary small amounts along ths paths 

of integration. 

J.l 8i CIS ICS T««»T 

Decision theory represents one of the most recent developments in operations 
research. Ifc has found considerable prior application as pert of communications, 
radar, and pattern-recognition ay sterna, ho-ever. Its applications to the actual 
decision proceae have been relatively recant and few. 

In decision theory two factors contribute to a decision! 

(1) The probability of the outcome# if a given decision is amd« 

{£) The value of the outcomes 
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The theory, then, attempt* .0 define the decision process In terms of a 
number of states, values being associated with each. Application of the theory 
results in identifying a beat course of action, generating alternative states, 
establishing new values, and providing a dynamic framework for the decision 
process. 

Ml CtST-ISTIMATIM lIMTMRMin INI CMFIOENC! INTEIVIIS 

3.10.1 Cost-Estimating Relationships 

When the system is complex, there is usually no simple two-parameter formula 
that relates system characteristics to system performance or system cost. How- 
ever, through the use of the statistical technique known as linear xegreaeion, 
the equations or relationships of interest can be approximated by a straight line 
or by a hyperplane — that is, a straight line o.n n dimensions. 

The first atep in the procedure la to establish a list of system parameters 
on the basis of engineering judgment; this list includes system characteristics 
that ars expected to contribute significantly to the variability of system per- 
formance or system coat at any time during the research and development, initial 
investment, or operating phases of the system's life cycle, Per example, if the 
system were a radio receiver, the Hat of system parameters would Include euch 
items as weight, volume, sensitivity, selectivity, signal roll ratio, atreea 
characteristics, and coat, 

Qreat care should be taken in compiling these liats, because the eese of 
computation and the adequacy of the resulting prediction depend primarily on the 
discrimination exercised at this point. The following are the usual priorities 
for parameter selection in the regression analyaes: 

(1) Parameters that are considered, on « cosc/snglneerlr.?: baaia, to have 
a second-order effect on the applicable coat cat-^ory are excluded 
initially. 

(<?; Parameters that exhibit l* ctle variation among the systems Sn the 
study are excluded initially. 

(3) Parameters that might be difficult to quantify during the initial 
procurement stages ha-.e lower priority than others. 

(4) Parameters that ere highly correlated »ith one or more other para- 
meter* have lower priority. 

'5) Parameter* ire selected so tf possible, et least one from etch 

of the following categorise is initially investigated: 

« session characteriat -o* 

. State -of- the- vrt character* sties 
. Scej&le&ity. or quantity chara-teriatle* 

, Effectivene** ch*r*-cteri*ti_e 
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One parameter may represent two of th* above characteristics. 

’’or example, a complexity characteristic such as rurix>r of active 
elements will often correlate well with reliability. 

(6) Where there la a choice between two or more equally important 
correlated parameter#, the parameter or parameter combination 
that la moat conducive to establishing trade-off relationships 
Is selected initially. 

The se.jnd step i„ he procedure Is to determine the coefficients of the 
regresrion equation. Vhe regression equation in Section 2~lk can be written as 

Xj »- A + BXg - CX 3 + m k 

where A, 8, C, and D are coefficients, or conspenta, that have to be determined 
and the X'a are the system parameters or combination of syscem parameters. Tht 
constants can be determined bj aolving ttie following set of equations simultane - 
ously: 

£X } - HA 4 B2X, ♦ CSX- + CEX^ 

LA,X,, - AXXg + BEXg 2 -r eXX^ + EEX^S 

ZXjX-j • AZX 3 e 82*2^3 + 0 * 3 * + 

TX^n - Ayjc 4 + K* 2 X 4 4 OX^ + DEl/ 
where N it the number of samples. 

Once the constants A, B, C, and D have been found ■ the regression equation 
is determined, and the value of tne multiple correlation coefficient, r, can be 
calculated from the following formula: 

r - B(.> x^-iXjXx^} •» ciiCXjXj- xXjiXjj 4 - .DjfcXjX* - M^x*) 

The value of the multiple correlation coefficient, r, will. always .be between 
0 and 1, and 1.; the value of r la 1, then si. the resale points lie on the plane 
(or isyperplane}. If the value of r 1* ray less than 0.7. then the sample 

potntu are net approximetsd by a plane, and ♦. us some other forts of a -predict less 
equatlor efeouid be tried. 

J.10.2 Conf vaen -?e In* trVtlg 

The next ares of interest in tbs regreeslob technique 1* the deteidilrtsttisn 
of t.se standard error of the e-xtlaate Shi th*. Interval «. Tits -sOTjor 1 . 


AMCP *06 19? 


(Ij) so -fete..' c ' tha aa&ple points shout th# plane ie Beaaured by the stands *d 
orror of th* sstiaate, £s 


S - 


°X 



(1 


■r ? ) 


where 


fl X 



i - <°J 

N (J*-l) 


2 


X » ffunber of variaolse used In predicting Xj 
X « Rusher oi‘ Mnplt point# 

r * uorr*latior oejffiolsnt 


Approximately 68 percent of the sanpl* points lie within ts of the plane 
dev* ruined by th* regression equation, and 95 percent of the ossple points lie 
within *23 (veesured in the X-^ direction } . 


Th* standard error of th* estisst* give* an lndicatior of t;* ap.<eal of th' 
o; iginal data points shout th* regression plena. Ho* ?v?r, when the regression 
equation la used to predict the coat of a new piece of equipment, the prediction 
interval or confidence interval is given by 


I “ iS 

1 * <*i-V <*j - 

V l,J-* 




eh* re 


X^ is the predicted v^l;e of X, obtained free the regression c-’a*ti 


ion 


3 la tU standard estimate of th# error 

t # - t st («/ 2 . n- 2 ) is s vsiue fr«s th« t diStribu - ion 

X i» th* value of thi paraasteys for the nee equipment 
2 is the near, or average cf t .*» x 1 a 

X is the number of par-meter# used in the predlctior *qu* 'on 
0 ^^ la se ** 7 i sifted below 

To obtain th* vs .use 5^,, th- following procedure is uv*l. ' * ' t pare* 
met**** uaal in prv •‘leting X^ are X- , Xy and X^, then th* firs? » f is to 
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calculate the quantities V, using the following equations: 

V 22 - NlKg 2 - (IX,) 2 

v 33 ‘ - &h )! 

?Hk - ^4 2 - (»4 > 2 

v 23 “ V 32 " “ ^3 

V S : 4 * v 42 * NZXjX/. - 

V 3 4 - V 43 - NZX^ - ZX^ 

Now, to determine the values for tJ 22 , Ug 3 , and U 24 , tht values of V are 
substituted In the following equations, which are solved simultaneously: 

V 22 U 22 + Vg 3 0g 3 + V 2 4 u 2 4 ** 1 


V 32 U ?2 + V 33 U 23 + ? 34 U 2k m 0 
V 42 °22 + ''43 U 23 * V 44 J 24 “ 0 


To determine the values for II- 


- 32 .’ u 33> 3 - lu «34» 

stltuted In the following equations, which again are solved simultaneous!:': 


and li- 


the values of V are sub- 


V 22 * J 32 + V 23 °23 + V 24 °34 * 0 

v 32 u 32 + v 33 u 33 + v 34 it 34 « 1 

v 42 u 3 2 4 V 43 °33 + V 44 u 3 4 “ 0 

Finally, to determine the values for U 42 , U 43 > and 0 44 , the seme values of 
V are substituted into the following aquations, which are solve*, simultaneously: 

V 22 U 42 + V 23 U 43 + V 24 D 44 " 0 

V 32 tJ 42 + V 33 u 43 + v 34 U 44 * 0 

v 42 ,J 42 + v 43 U 43 + v 44 U 44 “ 1 

The confidence intervals can now be found. For example, if the 95 percent 
confidence interval is desired, the values of S, N, X, and U are substituted in 
the confidence interval equation, along with the value of (based on a 95 „ar- 
cent confidence level and the number of samples, N). The relevant statement that 
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4 


■>N l 

VW;.'' 


can now be made la tha^ the estimated cost of the new equipment Is x^, and there 
la a 95 percent probability that the coat will be somewhere between the upper and 
lower confidence limits. 

3.10.3 Example: Simple Linear Regression (Two Variables ) 

Oiven the Information In tha following table, determine the cost for a new 
piece of similar- type equipment, the xyz-2, which has & volume of 30 cubic feet. 


Plaiting 

Equipment 

Volume 
(Cubic Pt.) 

Cost 

(Itollars) 

URC-32 

20 

10,392 

WRT-2 

34 

12,2?9 

B-390 

2 

1,096 

ORC-35 

3 

6,628 

QhC-S 

12 

3,307 

310-21 

14 

4,366 

SRC-20 

18 

7,588 

URT-1 

36 

14,580 

XYZ-2 

30 

X 1 


'The procedure to be followed consists Initially of linear regression, using 
the method of least squares. If It is found that the data are not essentially 
linear, then other methods are tried, such as logarithmic, quadratic, etc., until 
an appropriate prediction equation can be obtained. 

The fom of the linear equation la 

X x » A + HXg 


where X^ represents the cost and Xg represents the volume. 

The values of the coefficient- A and B can be found by simultaneously solving 
the fallowing equations. Since there are only two variables, the relevant equations 
(excluding any terms containing or X^,) are 

23^ « HA + BSU 

HC^ - A3X> + BZslJ 
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Prom the Information in tbo preceding title, 

IX X - lC,^i t 12,278 + ... ■ 60,335 
JXj - 20 + 34 + 2 + ... » 139 
IX - 20(10,392) + 34(12,278) +... - 1,411,620 
2X> 2 - (20) 2 + (34) 2 ♦■...« 3529 

N - 8 

Substituting these .lines Into the above equations gives 

60,33i> - 8A + B(139) 

1,411,620 - A(139) + B(3529) 

Solving theb. equations simultaneously for A and B yields 

A - 1840, B - 326 and 
x 1 «■ i84o 326 r 2 

Thus ths coat for an .‘quip lent with a volume of 30 is 
X 1 « 1340 + 326(30) « $11,600 

An alternate method of solution, which determines the value of the corre- 
lation coefficient r before solving for A and B, does not require the aiaultan- 
eoua solution of two equations. 

The cjATfc Jat-^on coef f icient , r, for two variables Is given by 


where 


IXjXj, - N 


* 1*2 



the standard deviation , of a number of semiJlte, N, is 



the mean or average of the N sample points, X, ie 
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Note,: the absolute value of the correlation coefficient will be somewhere 
between 1 and sero. It has been found empirically that an absolute value of 
r greater than 0.7 will yield an acceptable result; l.e., the data points are 
essentially linear. If the data points actually do lie on a straight line, the 
absolute value of r will be 1. A negative value for r Indicates a line with a 
negative slope, 

The values of A and B can now be calculated by using the following equations 



•A -T x - B°£j 

Tor this example, the information from the table is substituted into the 
above expressions to yield the following: 

IXj • 10,392 + 12,278 + ... - 60,335 
X^) 2 - (10,392 ) 2 + (12.278) 2 + ... - 605,554.500 

M 7,542 

O 

a X. - /8(60 5 ^g4 l 500lT'(60 t 3 35 l 2 „ 4 637 
1 v/ 8(8-1) ’ 3 

SXg * 20 + 34 + 2 + ... - 139 
xdg) 2 - (20) 2 + (34) 2 + (2) 2 + ... « 3529 
T> - If- - 17.4 

% - . 12.6 

^ sj 8(8-1) 

x> m r(gQXlO,392) + (34)(12,2f8) +..,3 - 8(7,54g)(l,7 t 4i . n ag 

8(4,637X12.6) 

B - JM2114§2IL - 326 

12.6 

A - 7542 - 326 (17.4) - 1840 

Therefore X 1 » 1840 + 326 Xg end as before, for an equipment with a volume of 30, 

X x * 1840 + 326 (30) - $11,600 
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Determine the standard error of estimate for the above data, and the 95 per- 
cent confidence interval for the xyz-2 equipment. 


The vertical scatter of the data points about the regression line 
X^ - 1S40 + 326 Xg is measured in terms of the standard error of estimate, S 


where 


S • Of 
*1 



It has been determined previously that 


therefore S - 4637 


0, - 4637, r - 0.89 
*i 

sjl - (0.89) 2 - 2130 


Note ; Approximately 68 percent of the data points lie within i S of the 
regression line, and 95 percent lie within ± 2S. Therefore, if a graphical plot 
of the data points is made, the parallel lines at a vertical distance of 2130 
from the regression line X^ *» 1840 + 326 Xg contain approximately 68 percent of 
the data points and the parallel lines at & vertical distance of 4260 from the 
regression line contain approximately 95 percent of the data points. Ae the 
sample size Increases, the number of data points within ± S and ± 2S would become 
closer to 63 percent and 95 percent, respectively. 


To calculate the confidence interval fo* the cost of the xyz-2 equipment, 
the following equation is used;* 


X 1 


, (Xg - Xg) 2 

J 1 + N-TraTa7F 

V et 


where 

t e is a value obtained from the t distribution tables 

Xg is the ordirate of the regression line for which the confidence 
interval is to be found 

Other symbols are as in the previous example. 


♦Note - This is another form of the equation developed previously and is also 
applicable for simple regression. 
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For the xys-2 equipment , 

X x - 11,600 
S - 2130 

t - (0.025,6) - 2.45 (from ”t H table for 95 percent confidence and 
8 data points) 

Xg • 30 cubic ft 
\ - 17.4 

0 Xg - 12.6 

N - 8 

The 95 -percent confidence Interval is 


11,600 t 2.447 (2130) 



- 11,600 * 5900 

Therefore, the estimated cost of the xyt-2 equipment Is $11,600, and t5^re Is a 
95 percent confidence that the cost will he somewhere between $17,500 and $5,700. 


This wide range for the confidence Interval Is quite large. However, the 
significant fact Is that without an Indication of the range of probable values, 
the decision-maker would have no feeling for the accuracy of any predicted para- 
meter. It la better, of course, to have a narrow range for the 95-peroent con- 
fidence Interval, but this can be aohleved only if additional supporting data 
are available. 


1.11 EXPIKKRC! CttVSfS 

There are several factors that can reduce the unit cost of an equipment as 
the total number of equipments purchased is increased. Two such factors are the 
initial tooling coat, which can be spread out over a larger number of equipments; 
and the coat of materials, which can be reduced for « quantity purchase. 

Another factor that can reduce the unit cost of an equipment (unrelated to 
the two factors above) is the learning curve, or experience curve T • al b , where 

T • cost to manufacture equipment X 
a m coat to manufacture equipment number 1 
X • equipment number 
b - exponent of experience- curve slope 
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Empirical data have shown that the experience curve la appropriate for 
predicting the costs of aircraft engines and airframes and several types of 
electronic equipments. Normally, the experience ourvee are developed by the 
equipment manufacturer. The experience curve Is based on the fact that as the 
quantity of equipments being manufactured la doubled, the cost to manufacture 
each successive equipment is reduced by a constant percentage. If cne epuip- 
went costa $1000 to manufacture, the seoond equipment la following a 90-percent 
learning curve, and the eighth, sixteenth, and thirty- second equipment aaoh oosfc 
90-percent less than the previous quantity. 

Normally, the cost reduction described by the experience curve it due 
entirely to the reduction In man-hours necessary to produce an equipment, through 
the natural process of man learning his job better by repetition. There are, 
however, fully automated production lines with experience curves baaed on the 
fact that as the production line Is operated, supervisors can develop Improve- 
ments and chort-cuts In the process. 

The mathematical method for fitting data to the experience curve Y - aX b 
is l->g-log least- squares regression. Once the constants "a" and "b" have been 
found by ualng the regression technique, the unit coat of any equipment can now 
be found. Por example, suppose, "a", the cost of the first equipment, Is 
$20,000, and the value of "b" Is -0.322*. The cost for the sixteenth equipment 
la given by 

Y l6 - ( 20 , 000 ) ( 16)' 0 ' 322 « $ 8,200 

Tables have been developed by several Army agencies that can be used to 
reduce the ca. nt of calculation for unit, average, and cumulative costs for any 
number o^* equipments with any slope. 

Another method that Is usually a good approximation for determining the 
experience curve Is the "eyeball" method, i.e., plotting the data points and then 
drawing a straight line through the spread of points with a straight edge. Graphi- 
cally, the data points are plotted on log-log paper, and If the relation Y - aX 
exists, then the data points will fall essentially along a straight line. The 
slope of this experience curve can be found readily from any two points on the 
straight line whose ordinate s are separated by a factor of two (one equipment 
quantity double the other), Por example. If It Is found from the curve the ninth 
equipment costs $500 and the eighteenth equipment costs $450, than the elope of 
the experience curve is 90 percent. To establish the values of "a" and "b" (in 
the equation Y - aX b ) from the g’-aph, note that a" represents the coat of the 


• The value of b will always be negative unless the unit cost for succeeding 
equipments increases. 
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first equipoant , and that "b" can be determined from the slope. The slope of 
the experience curve and the exponent b ere related by the formula 

% slope • (2) b x loorf 

which follows logically from the observation that the oost of the first equip- 
ment (for X m 1) it equal to "a" and, by definition, the oost of the second equip- 
ment is equal to "a" tinea the elope; therefore 2 b (for X - 2) oust be equal to 
the slope. 

i.it eMMCMmvm amirs's 

3.12.1 Qeneral 

There are two main area a of usage in which oost- sensitivity analysis is 
used. First, the Individual oost constituents should be checked to determine 
how changes in them affect the total oost. For example, a 5-peroent change in 
the maintenance oost of an avionic system may change the total lifetime oost of 
the system by 13 peroent, while a 10-percent change in the equipment oost changes 
the total lifetime oost by less than 1 peroent. The implication here is that the 
oost analyst should oonoentrate on refining the maintenance- cost prediction, 
whereas a relatively gross estimate of the equipment oost will be sufficient. 

Secondly, any assumptions that were made In the analysis should be checked 
to determine how changes in the assumptions sffsot the total oost. For example, 
if it was assumed that the equipment would be operated for 100 hours per month, 
the total oosts should be calculated for operating times of say 75 and 125 hours 
per month to determine the effeot of this assumption on the overall cost. Of 
course. If the total cost is sensitive to any assumption, the results of the 
sensitivity analysis should be shown to the decision-meker with the range of the 
assumed value Indicated. 

3.12.2 Qoat-Senaltlvlty-Analyals Problem 

The coat information given below for two alternative cosaunloetlons systems 
Is cased on regression analysis. 


Oost Categories 

Total System Oost 

(10- year lifetime) } 

Communication 
System A 

Communication 
Syatsm B 

RAD 

$ 10,000,000 

$ 12,000,000 

Equipment Acquisition 

300 , 000,000 

320 , 000,000 

Sparse and Spare Farts 

14,000,000 

8,000,000 

Initial Maintenance 
Facilities 

4 , 500,000 

6,000,000 

Publications 

500,000 

800,000 

Maintenance 

160,000,000 

120,000,000 

Annual Training 

1,000,000 

900,000 

Annual Facilities 

2,000,000 

2 , 500,000 
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The ooet figures repreMnt average (or expected) values, tilth som standard trror 
of estiaats. 0n« oost-estlaeting relationship was developed for aaeh oategaryj 
therefore, the standard err-r for each oategory la the saM for "rate* A and B, 
but the oetegorisr will generally have different standard errors. Xn other words* 
the standard errors are Identical horizontally, but not neoesaarlly vertloally. 

Question It Xn which oost oategory does oost uncertainty have the greatest 
lapsot on the oost comparison, and how great is this lapaotf 

Quart Ion 2i Are the results sensitive to the aesunptlon that the system will 
be In operation for 10 yearsf 

Answer to Question 1 

The oost totals for each system are: 

A: $492,000,000 
B: $470*200,000 

A, then. Is ostensibly wore expensive than £. However, beoause of the 
existence of standard errors, a coat- inversion aay be possible, so that B, In 
faot* is the aor* expensive systee. 

Iquipaent Aoqulrltlon and !talntenanoe are the ten biggest categories; 
between then they sc count for aore than 90 percent of the total aystea oost. 

Assume that if a 00 et- Inversion exists. It is due either to Acquisition or 
Maintenance oost errors (or both), the other categories being so asall relatively 
(in terns of money) that they are essentially oonstants. 

The diff erenow between aystea oosta is $21,800,000, TUis If at least one- 
half of $21*800*000, or $10*900,000 were simultaneously added to the total cost 
of B and subtracted from the total oost of A, an Inversion would result. 

Now, the total oost for aystea A oan be written se 

Cbst* - 32, 300:000 ♦ (I A + 1^) 

where 

* A • Iqulpaent Acquisition Oost 
•> Maintenance Oost 

She total «©et for sys;sa £ can be written es 

Ooetj • 50 , 200,000 ♦ (gg + Mg) 
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Tha probability that system B float* sort than ayet« A la given by 

p (oortj > OC#t A ; . p f 30, 200, 000 4 4 *,) s 32,000,000 4 d A 4 H A )j 

•»[<•» ♦ V - !'****> > l,800,000j- P £z> l,800,f'00 

Since the tr v ulated valuta of *A' Eg, Mg, and Kg were obtained by recreation 
technique*, the at valuta tie the expected valuta. 

Thua if 2 ta the random variable 

2 - (Eg + Mg) - (E A + Kg) 


then the erpeeteo value of £ la given by 

2 - (Sg + Kg) - (* A + M A ) - (320,000,000 + 120,000,000)- ^(300,000,000) 

+ (160,000,000) j 

or 2 - -20,000,000 

Ideally, the stand trd error* of estimate for E A , Eg, K A , and Kg (and the other 
cotta aa well) ahould have been fumlahed. since they have not, a "*or*t" caee 
eatiaate la obtained ae follow* : 


A a tune that the percentage error* in E and K do not exceed acae nominal 
value, aav 10 percent; thua 10 percent or E A - 30,000,000, and 10 par cent of 
Eg * 32,000,000. If these valuta are owuldered to represent three standard 
errors, which a Inoat guarantee* that the error will be less than 10 percent, 
then 

Sg - 10,000,000 and 3, - 10,700,000 


However, from the aseuBptio.^ that tha aaae regression equation was used, S_ 
aust equal Sg^, so that if Sg^ and Sg^ are "averaged". 


\ * \ ' 10 


,300,000 


Similarly, 


\ ' % 


- 4, 700, '-'00 


n * ns 2 " J4 A + 4j~* ^ " Viiuo.j) 2 ♦ 2(4.71*} X 10 12 

• y/ TVTTT X 10 6 

- 16,000.000 

3-*3 
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The protlea la shown graphically below. The curve represent* the distribution 
of equlpoent scqulsltlon end Min tenon oe ooete; l.e., the equlpaent acquisition 
and Mlntenanoe cost of sjrrtu 9 Is "expected 1 ' to be $20,000,000 leas than the 
eqrljasent acquisition and aalntenanoe cos- of. ays ten A, and the s-andard error 
of thla distribution la $16,000,000. The aoro line represents that point at 
which the equipment aoqulaltlon and Min .enanoe costa for A and B are aqtui. The 
$1,800,000 line represents that point at which the total wystaa costa for A and 
B will be equal. 



The probability that the cost lies to the right of the $i.,500,do0 line (a cost - 
Inversion) Is given by 



o 


where r» A Is the value of interest on the nonsal density function 
u Is the mean of the function 
oi* the standard deviation 


Substituting the values free the exaagle probXea into the sh^ve equations 
yields 

1,3 00,000 - CrgPaOpOaPQPi . i , 0 
l6.0CV,000 

which, fro* Tsble C-l lr» Appendix C, corretpenda to 0.0869. or sppnoxiaateiy 
** percent, fherefore, 'or an estlna-ed aajclmss errs- of 10 percent in the acqwl 
sition and aaint.rince cost, the probability that systea B U really acre expen- 
sive than syrtess A is 9 percent, if this process IS repeated for an estirated 
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ma rl m m er ro r of 100 peroant, the calculations lead to rite remit that the 
probability that *yst«s B la really more expensive tna.: system A la 33 percent, 

the answer to .ju»et -an 1 , then, is that uncertainty In the predictions of 
aqttlpaant acquisition and maintenance 00 at have the greatest intact on the cost 
coaparlacn; and if It la assumed that theaa predictions nay be off by ae such an 
100 percent, there la still only a 33 -percent chance that the coat of system B 
will be tore than the cost of eystus A, 

teaea. &£. toun&i .i 

Tor this example, assume that Maintenance , Tr ainio*, and Facilities are 
t?me-baaed ooet., and assume further that they are linear with tine (although 
any funct' :» other than linear could also be hand- sc easily with, for exanp t.e, 
a graphical solution}. 

Then ooste would be oategirlsed as follows: 

poets System A 

Fixed Oosta $329,000,000 

Annual Costs 16,300,000 

(per year) 

For y years, the cost of system therefore, will be 
Ooet A • 329,000.000 ' .,300,000 y 


System B 
$3*6,800, CXX) 
12 , 3 * 0,000 


Similarly, 


Coetg - 3*6,800,000 ■+■ 12,3*0,000 y 


Thus Coet^ i Cbetg whe.: 

329.000,000 -*■ 16,300,000 y s 3*6,800,000 ♦ 12,3*0, COO y, 
or when y*~ *.5 year* 

the answer to Question ?, the*., la that the remits ere not #*r.oitlve to 
the assumption that the system will be operated for 10 year* -- that 1*, the 
ooet of eyate* 8 will be lower than ths cost of system A, unless t‘«e system Is 
to be In operation for leas than *.5 yeirs. 

(bote very careful y tne assumption of ;ineerSt>, 1.#., ‘ha*, the “ost is 
directly presort lone, to tlaa. This assumption, fet* specific systrs*. a*/ cer/ 
•ell nc ‘ te true} 


3 -* £ 
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BASIC MATHEMATICAL ANB STATISTICAL CONCEPTS * 


4.1 INTRODUCTION 

Basic mathematical and statistical concepts are reviewed in this chapter. 
Topics include algebraic principles and formulas, the various types of probabil- 
ity distributions, and procedures for statistical estimation. 

4.1.1 Preliminary Definitions 

Some of the principal terms used in this discussion are defined as follows: 

Random Outcome. The vilue of an empirical observation that ca.mot be 
predicted (lack of deterministic regularity) but has statistical regularity in 
that the value has a relative frequency of occurrence in a merles of independent 
observations of the phenomenon (the result of tossing a die, the tine-to-f allure 
of a device, etc.). 

Trial. An action or experiment that yields a random outcome (tossing 
a die, life-testing a device). 

Independent Trials. Trials of which the outcome of one has no effect 
on the outcome of others that follow. 

Event. A set of outcomes. The event has occurred if one of the out- 
comes of the set is observed on a trial- (If the event is ar even number on 
*he toss of s die, it occurs if the number 2, 4, or 6 is observea) 

Independent Events > Sets of outcomes based on independent trials. 

Mutually Ercl”3lve Events. Two or more events that cannot occur 
simultaneously (odd and even numbers on one toss of a die). 

4.1.2 Notation 

The ollowlng probability notation applies: 

P(E) = probability of event E, where 0 i P(E) s 1 
P(E) = probability of event "not E" = 1 - P(E) 


♦Some of the material used in this chapter was developed by ARINC Research 
Corporation for the U. S. Weather Bureau under Contract Cwb-11349. Reproduced 
by permission of the U. S. Weather Bureau. 
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P{2 1 + Eg) * probability of events or Eg 'or both If they art not 
mutually exclusive) 

P(EjEg) - probability of both events E^ and E g 


4.2 KFIIOTIMS IF MfMMUTV 

4.2.1 Classical Definition 

The classical definition of probability is ac follows: 

If an experiment can result in n different, equally likely 
and mutually exclusive outcomes, and if r of these outcomes 
correspond to event S, the probability of E, denoted by P{E), 
is the ratio 


P(E) - r/n (1) 


E xample : If a card is drrwn at random from a full deck, 
there are n"=> 52 mutually exclusive and equally likely out- 
comes; r » 4 of these are the evert of drawing a kin?. The 
probability of drawing a king, from Equation 1, is 4/52 - 1/13. 


The classical definition of probability is one that involves an a priori 
evaluation and is useful only if all the possible outcomes can be enumerated and 
are equally likely and mutually exclusive. "Equally likely" can be described 
as the lack of any bias favoring one outcome over another in a trial. 


4.2.2 Relative-Frequency' Definition 

The relative-frequency definition Is as follows: 

The probability of an event ia the limiting ratio of the 
rumber of outcomes favorable to an event (r) to the number 
of .trials performed (n) as the number of trials approaches 
infinity. If n is large, the ratio r/n can be used to 
estimate the probability. 

Ex ample : In the life-testing of 100 devices, it was 
observed" that 15 failures occurred before 20 hours. The 
estimated probability of failure before 20 hours is there- 
fore 15/100 ■ 0.15, The relative-frequency definition of 
probability requires a statiotical estimation involving 
valid experiments and sufficient data to yield an estimate 
that is fairly stable. 


4.3 AllIIR Alt MIISIPUS Wl FRRWUS 

The following algebraic principles and formulas aim useful for applying the 
classical definition of probability in those cases in which it is valid. 


4.3.1 Two Basic Counting Principles 

The two basic counting principles are as follows: 

(1) If event- A can occur In "a" ways, event b can occur in "b" ways, and 
both can occur together in "c" ways, then A or B or both can occur In 
( a + b-c) ways. 
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Exewale ft : Sped* or heart In on* draw: A * spade, 
B * heart. 13, b • 13, c » 0. A or B in 13 + 13 « 
26 ways. 

gSygjL*.b: Spade or ace in one draws A - spade, 

B - ace. a - 13, b - 4, c - 1. A or B In 13 + 4 - 1 ■ 
16 ways. 


(2) If there ar* ”a" ways of perfoiming the first operation and "b" ways 
of performing the second operation, given that the first operation 
has occurred, there Is a total of a x b possible ways for both 
operations . 

Example a s Throwing an even number on a die and 
drawing an ace from a decks a - 3, b « 4. Total number 
of ways - 3 x 4 - 12. 

Example b s Drawing t\;o spades from a deck of cards s 
a ■ 13, 6-12. Total number of possible ways * 13 x 12 * 

156. 

4.3.2 Permutations 


A permutation is a particular arrangement of a collection of objects . The 
total number of permutations of n different objects is 

P(n, n) * n(n - 1) {n - 2) ... 3 x 2 x 1 (2) 

« n! 

The total number of permutations of n objects taken k at a time is 

p < n ' K) * Tn " - ! V 7 ! (3) 

The total number of permutations of n objects.- k^ of which are alike, kg 

r 

of whleh are alike, . . . , k p of which ere alike { kj » n | is 

1-1 



(4) 


(Note: 0.' is defined as equal to 1.) 

Example as For the letters ABODE, there are 5! « 120 

■' • n ' £ ! 

permutation*. There are fg - jn « 60 permutations if three 
of the five letters are to be selected. 
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Extacis b i In the word SUCCESS, there le one C (k, - 1), 

one 2 (kg - 1), two C'e (k^ - 2), end three 8’s (k.^ * 3 }. 

Proa Equation 4, the total number cf possible pe nutations of 
all. seven letters In the word SUCCESS 1* 

infe r " 420 

4.3o Combinations 

A combination is the number cf ways in which k out of n different items 
can be selected without rcjard to order, symbolised by 

(£) or ( 5 ) 

where 

/n* . a: 

V’ ITTn DT 

Example ; Por a unit composed $f five components, at 
least three must be successful. Bow many ways can the unit 
be successful? 

Pros Equation 5, 

( 3 ) + ( ? } * { 5> * * S?iT + 

- 10 + 5 + 1-16 

4.3.4 Bas^c Probability Laws 
4. 3.4.1 Addition Law 

If A and B are two mutually exclusive events, the probability that either 
of then will occur in a single trial is the sun of their respective probabilities, 
or 

P(A ♦ B) - P(A.) f P(B) (6) 

In general, if there are k mutually exclusive events, 

P(A, ♦ ^ +..'.♦ Ajj) - P(A^ ) + P(Ag) + . . . + FlA*} (7) 

It the two events A and 3 are not mutually exclusive, the probability th*t at 
least one of them will occur is 

- P{A) + P(B) P(A8) 


4 -4 


P(A + S) 


( 6 ) 
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For three non -mutually exoluslve events, 

P(A 4 B + C) - ?(A) + P(B) + P(C) - P(A5) - P(AC) - P(BC) 4 P(ABC) (9) 

Ths soot gsnsral for* of ths addition law staves that tha probability of an 
•vant is tha stsa of its mutually exclusive forms. It it ir assumed that A and 
B ara not mutual ly axolusiva but that tha ;iree events (A and 3), (A and not B), 
and (B and not A) ara mutually axolusiva, than 

P(A + B) - P(AB) + p:aS) + VCRB) (10) 

4. 3.4. 2 Multiplication Law 

Zf avanta A and B ara independent, tha probability of tha compound event A 
and B is equal to tha product of thair respective urobabili ries, or 

P(AB) - P(A) P(B) (31) 

The cxtanslon to more than t«ro avanta follows directly. 

4. 3. 4. 3 Conditional Probability 

If events A and B ara not independent — i.e., tha occurrence of one affects 
the occurrence of the other — than eorOltlonal probabilities exist. Tha con- 
ditional probability of A given that B haa occurred la denoted by P(A|B); sim- 
ilarly, the probability of B given that A haa occurred is denoted by P(B|A). If 
events A and B are not independent, 

P(AB) - P(A) P(B|A) - P(B) P(AfB), (12) 

which reduces to P(A) P(B) if A and B are Independent. 

For three eventa, 

P(ABC) - P(A) P(B|A) P(C|AB) (13) 


Aleo, 


P(A|B)-f^ (14) 

P(B|A)-^j|i ('.5) 
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Squat Iona 14 and 15 lead to a fom of aaw 1 Tfiaara^. 

P (a!b) - **4 ( 1 ^ * - p(A) P(8$ !§}$ FT B| T ) (l6) 

Xn thia ^plication, P(A) and P(X) art uaually a priori probabilities of tha 
event* A and not A. It la neceaearp to aodlfp P(A) on tha baala that erant B 
haa oeourrad in acoa experiment vhooa outcome la known to ba lnfluanoad bp A, 
aa refleeted bp tha terae P(B|A) and P(B)X). 

Exeaple i Aaauaa that a box of 100 outwardlp indlatingulahable 
parta la ooapoaad aa followat 


Quality 

Manufacturer 

Totala 

A 

B 

C 

Oood, 0 

40 

27 

3 

75 

Bad, ff 

10 

3 

13 

25 

Totala 

50 

30 

20 

100 


Tha following prooabllltlaa art baaad cn tha elaaale definition 

»w • ***■ 1 

and on tha probability lava dlaouaaad abova. 

CaaaJLt For a randan aalactlon tram tha bos, what la the probability oft 

(a) Drawing a part aanufaotured bp A? Proa Equation 1, 

r{A) "" 1 $ * £ 

(b) Drawing a bad part? 

(c) Drawing a part manufactured bp C which la alao bad! 

P{(SI} *» -jjjp * O'* 14 

(d) Drawing a bad part aanufaeturad bp C, glean that a part aanufaeturad 
bp c waa selected* 



AMCP 70S*1tl 


Counting indicates that there are a total of 20 possible ways for selecting a 
C part, and In 12 esses bad parte will be seleeted) hence, froai Squat lor 1, 

P(V|C) - ^ - 0.60 

Similarly, fra Equation lb, 

*(S|C) - - M - °‘ 60 

Case 2 t If one part Is drown randomly. Equation 6 yields 
F(A ♦ B) - P(A) + P(B) - - 0.8 - l-P(C) - 1 - $ - 0.8 

Proa Equation 8, 

P(A + ff) - P(A) + f(B) - P(S ff) - J + i “ - 0-65 

Counting Indicates that the number (A or IT, - 50 ♦ 25 - 10 - 65. The number of 
possible outcomes - 100. 

?{A + 5) - 0.65 

Case 3 i If two drama are made, what la the probability that both ltetm 
selected are manufactured by A, for the following: 

(a) Drawing with repla c em en t. Independent event*? 

«v»> • 

• iS t J8» * !/ * 

or, from Equation 11, 

KAjV) - P(Aj) P(Aj) - 1/9 * 1/i • 1/b 

(b) Drawing without replacement, dependent events? 

r( *lV } * tS 'w' 8 “ T9$ 

or, from Equation '2. 

p(*i V - P(^) -$*$?- ig| 
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Case A t What U tte probability of seleeting * part > wteX by 
aanofaoturar A on fcho first dree on* a bod its* (?) on tha teoond dravt 

(o) With replaoenenti 

t(A^) « P(Aj) F (ty ■ 1/8X1A* 1/8 * 0.1250 
(0) Without rsplaosaant: 

F(Aj^) - F(A l ) Ffl^) 

Sloe* Aj (A on first draw) consists of tho assets A^0 1 (A and 0 on first 
dies) or Aj7j (A and 7 on tha first drew), than 

+ , < A i 5 l ) F t^l A i af i> 

- (#*(#♦«<# 

- 0.1253 


too Identical boxes as follows! 


[ Manufacturer A j 

0 

7 

AO 

10 

Total 

50 


that tha porta ara distributed in 


Vat mtr. A (7) 

0 

7 

35 

15 

Total 

50 


If a boot Is c h os e n at rondos and a part la as looted free tha bos. shat is tha 
probability that tha pert is esnufaetured by A if It la found to be good? 


The a priori probability if choosing a part esnufaoturad by A is F(A) * 
l/fil that of t boosing a pert esatfsefcured by not -A la PfX) • 1/S, since the 
boas* art idt^tloal end one part is rsedoaly chosen. Free Sages T h s or s * 
(Aquation 1 6 ), the probability that box A la chosen, glten tha. e good lies Is 
selected, is 


1 >. * 3 

?(A|0) • F m F rf tf ) ’PW H “T(7p r y * ^ * 0,5S * 
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MoMi by Bay** T’eoran, the prior probability that a part 
■anofaetured by t mo ohoeen la mdlfted fra 0.5 to tha 
poatarlor probability of 0.553 oo tha baa la of tha lnformtlon 
that a pood part vaa aalaotad fro* tha choeen box. 

*.3.5 Application of probability law to Reliability 

In aoat of tha appllaatlcaa of probability thoory to ra liability work* tha 
omsta In qoaetlen ora axpremed io tnaa of a him variable; hanoa. tha probe- 
bUltloa thmaalma ara not oonatanta but a ra functlone of tlm, denoted by t. 
ft* above fonatlaa bold equally mil whan Interpreted aa functlone of tlaa. 

Am an axmpla, tha mUabUltr* at tlaa t la equivalent to a probability of 
no fallom bafora t. If m bam two apnlpaanta, a and b, tha probability that 
both oparato la, by tha aultlplleasion Xm (Bqoatlon 11) and undar tha aeeueptlon . 
or umptnuincti 

*ab < fc ) * V*’ * % (t) 

a .ara R^ft) la tha reliability at tlaa t of aqulpmot i. 

If thara ara two aqulpaanta, a and b, and if tha e/ata* la aueeasaful at 
tlaa t If althar a or b or both ara operable, than by tha addition law (Equation 8;, 
tha ayotaa reliability \(t) la 

%(t) - \lt) * yt) - R^ft) 


b.b.i Daflnitlooa 

Tha following daflnttlona ara pertinent to a diaeuaalon of probability 
dlotrlbutlona or daaaltlea: 

Bardo* Tar lab la , A quantity, x, for which — for every real number C-- 
thara exlata a probability that x la last than or equal to C. 

Dlacrete Batadoa Varlabla . A randua variable that can taka on only a 
fin'ta or coon table number of distinct vrluee. Tha random varlabla "number of 
failure within tha fixed elm Interval to, tl * la dlaerata. 

Contlnu m* lUrdo* Tartatl,# . A rv*i<m variable that can taka on any 
value within an interval (equivalent to taking on any one of a noo-der ume rab I a 
infinity of valoaa). Tha randaa variable ’tlm to failure* la contlnuoua. 

todafeUl&L flgftHte Jtmlim • ' mthmetical function, .ay f(x), 

which, for dlacrato random variable*, pi *, tha probability that tM randaa vari- 
able equal* x. for continuous random variables, f(x) give* tha probabl'Uy that 

x llaa In an interval, aay (a, bl, fra* the equation p (a < x * b] • j f(x)dx, 

J ® 
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Tho probability that a contlauooo rmt&m rariablo will Uki on a opoelfle vain* 
lw 4tflMi to bo aovo (o.g., tbo probability that a da riot will fall at taetli 
96.000 . . . bouro aqnala two), Intorralo rathar *ho«i potato wu»t bo eonalAtrad 
for tho ttaoal eontiwmu. caaaa . 

flWtfUl n fltfrthttqR TUSSlm- ** oatbMntleol fwwtloo that 
tapr oot — tho probability that tfao rondoa varloblo < 10 loot than or o-ptal to 
aoaa valoa aa dotorwlaod frao tbo probability Owwlty function. 

If X r r rto o apta a glwta walaa of x, than for a liter# to rondo* oarioblo wheat 
lowor Halt la L, the ouanlatlv* dlatrlbetfon function, F(X), la 

X 

y(x) • £ f(*) ( 17 ) 

x»L 

for 0 contimooa rondca variable, 

»(X) - f 1 (18) 


*.*.2 Frcoortloo 

Zf x roprooo.ito tla rondoa variable, «nd f(x) rwproaonto tho probability 
density function of x, than f(x) haa tho to.lowln* proporttoc: 

(1) f(x) » 0 fo- .11 x 


( 2 ) < 




• 1 


If x *o dloeroto* 


( 19 *) 


£f(*)d* - * — 


If x la continuous** 


the probability that x will to Xt on o value la tho interval {«» *>] la ( igp j 


f i* < x e bj 


Y f(x) if x la dlecroto 

r* 

f(x)l* if x it continuous 


z roproeonts tho ouat or or iU peealbio dlacroto valuta of x. 
x 

*TTv» llal to -• ta * oppiy if f{x) U —final to bo oquoi to ior« for ail 
Lepooolbio valuoo cf r. 

a-io 
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Ail ctMulatlve distribution functions hare the following properties * 

F(X) * 0 tor *11 X 

f(t) • 0 for *11 X < L, the lower Halt of the range of x 
F(X) • 1 for *11 X » 0, the upper Halt of the range of x 
F(X^) » F(Xj) If Xj * Xjj hence, the emulative distribution function la 

aoootonlesllF increasing. 

ft. ft. 2.1 gxasple A 

The function f(x), shown graph- 
ically In Sketch A and stated a* 

1/2, r - 1 


0-T5T 


0.5<)t 


0.25T 


J I 


Sketch A 


1 .CXI 

o.?5 

O.30 

0,23 


• 

i 

i 

-L - 
l 


Sketch B 


i/ft, x 
f(*) - x 


- 2 

- 3 

i/8, x - ft 
0, otherwise 


la a discrete protean: ty density 
function, since f(x) i 0 for all x and 

ft 

£ f(x) - 1.0. 


x-1 


The cursilatlve function Is 


fl/2, x - 1 

. . J 3/ft, x - 2 
p{») - < 

'■ 1 !?/0, X - 3 


1, 


. ft 


which, when plotted, yields ;he atep- 
funct lor. shown In Sketch 3. 

The proa ability that s Is greater than 
1 and les* than or equeJL to 3 Is, trow 
Equation 19a, 

3 

y f(x) - r( 2 ) * r{ 3/ * I/* ♦ i.-8 - 3^8, 

x=$ 

or Is equal to the probability that x 
la less thar. or equal to 3 alms the 
probability that It Is less thar or 
equal to 1 -- ;i*ae'y. 


si 3) - r(\ ) * t - : 


■2 *4 
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Sketch T 



X 


4. 4. 2. 2 Example B 

Th-j function f (x), shown graph- 
ically in Sxetch C and stated as 


f(x) - 


J 2 x, 6 * x * 1 
]_ 0 , otherwise 


lc & continuous probability density 
function, since x Is a continuous vari- 
able, and f(x) * 0 for all x, and 



2 xdx - x‘ 


] 0 l 


■» 1 . 


Tne cumulative distribution Is 



2 xdx - 


..2 

*• 3 


which io plotted In Sketch D. 

The probability that x is between ar.y 
two values In the range of x, say 
between a and b. Is (from Equation 19b) 



f(x)dx - F(b) - F(a). 


Thus the probability that x will be 
between 0.3 and 0.6 Is 


Sketch D 


P [0.3 < X SO. 6 ]- F ( 0 . c. , - F(0.3) 
- ( 0 . 6) 2 - ( 0 . 3) 2 
- 0.27 


* • u 1 Parameters and Moments 
4 . 4 . 3,1 Definitions 

A parameter la a constant that appears in the probability density function. 
It la more generally defined as ocrae measurable characteristic of the population, 

ouch aa the mean or range. 
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A moment, a doaortptlve proparty if a probal Viity density function, is 
defined u follows: 

(1) r th moment <tbout ssrot 


£ x r f (x), for discrete variables 
x 

£ x*f(x)dx. 


for continuous variables 


(2) r th moment about point "a r t 



a) r f(x), for discrete variables 


J (x-a) r f(x)dx ; for continuous variables 


(20a) 

(20b) 

(21a) 

(£ib) 


The first accent about sero or (i) Is the aa ap of th» distribution and Is 
a measure of central tendency. Mathematically, the mean Is the expected or aver- 
age value in the population; It is defined by the equation 




^xT(x), for discrete variables 
x 

J xf(x)tu, for continuous variables 


(2S>*) 

(22b) 


•V . 

The second moment about u Is callbd the variance, denoted usually by a 2 
(Ugln the previous notation). It Is a neasura of dispersion about the mean. 
Mathematically, the variance la the expected or average value of the square of 
deviations of all possible values from the mean; It la defined by 


a 2 *< 


^ (x“u) 2 f(x), for discrete variables 

/ m n 

(x-p.) i (x)dx, for continuous variables 


(23*) 

(23b) 


The greater the varisnee, the more variability there is In the distribution. 

The square root of the variance is known as the standard deviation . 

4.4. 3. 2 Relationship of Parameters and Moments to Reliability Theory 

To relate the above concepts to an important area of reliability — namely, 
the time-to-failure density function and the reliability function — let t denote 
the random variable time-to-f allure and f(t/ the time-to-f allure probability 
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density function, (f(t) ■ 0 for t < 0). The reliability over a time Interval. t, 
denoted by R(t), Is, by definition, 

R(t) - probability that failure occurs after t (the reliability function) 

- 1 minus probability that failure occurs before t 

r ^ 

- l minus f(tf) dt' (t'is simply a dummy variable of Integration) 

- fi'M «' 

Since the derivative of the cumulative distribution function le the probability 
density function for continuous variates, 

«*> ■ ^ , 

then 


The probability that an item will fail within the time Interval t^ to t g Is 
equal to the probability that it will fall before t g minus the probability that 
it will fail before t^; or, from Equation 191, 

p < t < t s J - jT t2 f (t)dt 

-p(t 2 ) - p(t x ) 

- [l-R(t 2 )j - [l-R^)] 

«. R(tj) - R(tg) (2U y 

The mean time to failure is, from Equation 22b, 

U - f t tf (t)dt, 

which for most density functions is equivalent to 


U- f m R(t)dt 
Jo 

The variance is, from Equation 23b, 

o 2 - f (t-u) 2 f(t)dt 

J 0 


( 25 ) 
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For reliability problems. the following definitions are Important: 

Wean Life . The first moment of a tlme-to-f allure density function — i.e., 
the average (in the sense of arithmetic mean) time that an item will function 
satisfactorily before failure. 

Mean Time to Failure fMTTF) . The term often used for the mean life of non- 
repatrable items. 

Wean Tins Between Failures fiPEF) . The term often used for the mean life of 
repairable items. 

(Note: The reliability for a time period equal to the mean life varies with the 
type of failure distribution; e.g., the reliability at the toagn life of a normal 
failure-time distribution Is 0.5, but It is 0.37 for the exponential distribution.) 

Median Life . The time Interval for which there is a 0.50 reliability (e.g., 

50 percent of items that have been life-tested would be expected to fall before 
the median life and the other 50 percent would be expected to fall after the 
median life). 

Failure Rate . The rate at which failures occur per unit time In the Interval 
t to t + h, defined by 

Mt; h) - (26) 

(Note: The term "failure rate" can be con. tain? because it Is used In various 
ways. It sometimes represents tne expected pi'oportion of failures In an Interval, 
provided all failures are Instantly replaced -- especially in connection with the 
exponential distil button, which le discussed in a subsequent section. Sometimes 
the term Is used to mean the conditional probe! illty of failure during an Interval, 
given survival at the beginning of the interval. In which case the divisor h In 
the above definition is omitted. In addition, the term 1 b often usod to signify 
the instantaneous failure rate or hazard rate as defined below.) 

Hazard Rate . The Instantaneous failure rate, defined as follows: 

z(t) * 11m \ (t; h) 

h-*o 

- < J7! 

Note that R(t) can be show:: to be equal to the fallowing expression: 

f l z(t)dt ( 28 ) 

R(t) * e J0 
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4.4.4 niacrets Probability Distribution* 

4.4.4.] Hresresgsvtrlc Distribution 

Ths requirements of the hype rgeoete trio distribution art as follow i 

(1) Thsr# are only two posslblt outcoiMS — o.g., success or failors, 
defective or not defective. 

(2) There le a finite population else. 

(3) Stapling le performed without replacement (dependent trials). 


Aseuee that a aaeple of n 1cm 1b drawn frow a population of else X that 
contains Mp eucceseee (an Integer) end X(l-p) - >4 failure® (an integer). The 
hype rge one trio probability density function give* the probability of obtaining 
x failures end (n-x) successes In the sanple. It Is expressed e# follows: 




* - o, 1, 


.» a 


(29) 


The emulative distribution is 



0 , 1 . 


(30) 


If N le large, so that the ratio n/X le wall (aay § < .05). teypergeoaetric 
probabilities can be closely approximated by the blncwlal probability distribu- 
tion, discussed below. 

Xxawle: Lots of 30 lteas each hsvs experienced er. 

•veraie portion defective of 20 percent, or 6 defectives 
Thus, Xp * (30) (0.20) *• 6 , Xq - (30) ( 0 . 80 ) " 24. ^If 5 
lteas srs Maplad frost a log, whet it the probability of 
getting (e) exactly 2 defectives?, (b) 2 or fewer defective#?, 
end mm than 2 defectives? 


(e) Exactly two defectives: 
From Equation 29. 
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(b) Two or fowor d»f ootiwsi 

F 1 2 or lets defectives] - F(2)j fro* Squat ion 30, 


« *1 • z 


2 »\ {6v-a\ rw» 


(i) d) 


• 0.296 + 0.447 + 0.213 
- 0.958 

(c) Pore than two defectives t 

F [nor* than ? defectives] - 1-F [2 or laoa defectives] 

• 1-F(2) - 1-0.958 - 0.042 

4. 4.4. 2 Binomial Distribution 

The requirements of the binomial distribution aro aa follows: 

(1} Thart ara only two poaaibla outcomes, success or failure. 

(2) Tha probability of each outcome la constant for all trials . 

(3) The triala are independent (equivalent to sailing with replacement). 

For n trials with constant probability p for auecaas and (1-p) for failure, 
tha probability density function for obtaining x successes is 

f(x) . (x)p* (l-p) n ” x , x - 0, 1, 2, ..., n (31) 

and the probability of x or fewer failures 1* given by the cumulative distri- 
bution function 


x 

F(x) - £ (kV(X-p) B_k , 


x - 0, 1, 2, . . 


whart 


(I) " srn&rrr 


a: - *(k-l)U-2) ... (3)(2)(1) 


0 ! » 1 


Tha mean number of failure# 1» np, end tha variance is np{ 1-p) . 
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i Uscele i Amism that past experience ladieetee that 
parte produced from a continuous production amui /laid 
5 percent defective. in a rondos sample of 30 parts (30 
trials), what is the probability that 2 or facer defectives 
will bo found? 

The following Information Is available t 

5s?s.*: : ss 

KKffiS s ass : ; : l%‘ °- 95 

Staple else, n, is 30 
IHnoe, froa Equation 32, 

P [k a 2) « P (k • 0, 1, or 2} - P(2) 


G 


■ I (£) »v-> 

k-0 

- (o.05)° (0.95) 30 + (O.Ot,) 1 (O.cij) 2 * 

♦ <°*° 5) 2 ( 0 - 95) 88 

- 0.812 

k.k.k.3 Poisson Distribution 

The Poisson distribution can be used as an approximation of the blwmlal 
distribution or sa the distribution of nunber of independent occurrences in a 
oontinuua, such ss tins, length, or volume. 

For an approximation of the binomial, the conditions are: 

(1) The binomial lew eppliee. 

(2) The eesple site, n, is large; and the probability of failure, p, ie 
email. A practical rule of thumb ie p * 0.10 and rp a 10, 

.r«e probability density function is 


f (x) ■ * " * (x * o, p > o, n > o) 


The parameter np reyreeente the expected or average number of f vlluree in n 
triale. 
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o 



Mat facia i Assuan that a saapl* of 2$ ltai*. is sslsctsd 
from % lsrfs lot in which 10 psrcsnt of tbs itsaa art 
dafsctlws. What Is ths probability of too dafsctlsaa In 
ths ■ asp 1 st 

Proa tbs Poisson spproadasfclon to tbs binomial, 

, w . . 0.2565 

Ths blnoaial probability Is 

f( 2 ) - ft) ( 0 . 10 J 2 (0.90 ) 23 - 0.2659 

If ths Poisson is aaploysd as tbs distribution of tbs nnabor of indspsndsnt 
oocuRMneos In a continuum, such as tl as, length, or voluas, ths conditions aro: 

(1) Ths nuabsr of cxpsetsd occurrences (say success**) p«r (Iran segaant 
of the contlnuua (a.*., an tntsrral of time) is a constant. 

(2) Ths nuabsr of ooourrsness produesd In any aubeessent is Indspsndsnt 
of ths nuaber cf ooourrsness in any otrttr subsegment. 

(3) Mo aeanlnc can ba ascrlbsd to tbs of non-oc currsncsa j e.g. , 

ths nuabsr of telephone calls not aads during a day, or ths nuabsr 
of non-defects in « shsst of stssl, canrit bs eraluatsd . 

If a la ths expected nuabsr of occurrsncss psr fl van segment of ths contlnuua, 
ths probability density function Is 

f{x) • (3*) 


Both ths Man and ths variance ars equal to a. 


Kxaapl*: Asauas that an Itaa will experience an 
awers#A dr 1 fallurss psr hour if •»'h failurs is instantly 
rspalrsd ur replaced. it Is desired to rind ths probability 
that r failures will occur if this ttsa is llfs-tsstsd for 
t hours and fallurss are rspalrsd or rsplacsd w* th identical 
Items. 


If > is ths evsrag* nuabsr of fallurss for otw hour, 
than a • At la ths average nuabsr of fallurss for t hours, 
fiance. If x rwprssent* ths nuabsr of failures (occurrsncss), 
frea location 3 b; 


I 

1 

f 



f(a) 



* * 0 , 1 , 2 , . .. 
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If n lteaa ere placed or toot, tbo mrifo mater of failures is ■ • nAt, 
or 

f(«) • , x - o f 1, a, ... 

m* 

If x • 0.001 psr boo?, t - 50, ud 10 itsas trt pot on toot, then a - nAt • 
10 (0.001) (50) • 0.5. The probability of oteorriaj too failure* to 



t{2) 



- 0.076 


*.b.5 Continuous Distributions 

fc.b.5-1 Ixpouentlal Distribution 

Tte probability density of tte exponential distribution Is glean bp 


f(t) 


*.-*< . 


. » , 


t * 0, 9 > 0 
X • l / 9 


(35s) 

(35b) 


■here 


ten •$ 

Variance •»* 

Tte exponential distribution Is prtesrllp used ss s fowls for waiting 
tines, or, in reliability, so s f wauls for tte tloo-to-fsllar* density function. 
Tte 1st tor uso Is s direct eoaseouence of essunlng that tte probability of fall- 
uro In tte Interval t to t ♦ h, given suttItsI to t, is a function only of h, tea 
length of tte lntorrsl, and la Independent of tte ago of tte product, t. This, 

In turn, tepllsx that if a dories fas not failed after seas period of op* -at Ion, 
It la aa good aa new, which la equivalent to tte eta tia m n t that tte hasard rata 
of tte exponential la a constant that equals tte reciprocal of tte asan life, 
usually denoted by A. 

Tte reliability function la 

*{t) - j" f(t)dt 
- •-** 

- e' U (36) 
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At th* mid Ilf*# •, th* reliability 1 * 


R(t - 0} - - ** x - 0.368 


Oivaa til* reliability vmr a tla* In t* real, the Man Ilf* can b* found fro* 
th* •qua*’ Lon 


ToS^JPT 


flfur* b-1 alioM th* a a p ooan dal reliability function with tin* gl*tn ir. e unit* 
A abort tabl* of tb* *xpon*ntial la (Iran in Appendix u. 


t {• unsea} 


m tmatmc »umm natmi 


Tb* exponential, |«u, and Foiaaor, distribution* ar* r*lat*d. For e>* 
Folaaco attribution, the rmnda* varied* la th* m»b*r of failure* ir a (1 v»r. 
tl»a Intarral . For the p»t distribution. th* ranlo* wiablo la tfa tin* to 
WM nth failure . For the exponential distribution, aa a apodal eaa# of th* 
*■**•. th* rando* variable la th* tine to the First failure. >ols»on prc-*a* 
la a tar* u»ad to eneOMpaa* th*** situation*. 
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fh* secnanlea underlyln* t he exponential reliability function in tost the 
haaard ra te (or tea conditional probability of failure in an interval, plvan 
survival at tbs beplanlnp of the interval) in lndtptnAnt of ten accumulated 
life. 

Tha um at thin tjjn at *fallam law* for complex »y»t— la naonlly justl- 
find because of the uojr foroaa that ana not upon tea ays tan «d prodoc failure. 
Different 4a tart oration dsahanlma, di ffa ra nt part liasand-mte funehictrt, and 
varying erv.romantal oonditiona often rant It in atmas-stmngth combinations 
that produea failuma randomly in time according to tea axpooantlal fallnm law. 

Anothar Justification for tea exponential in loop- life complex syatana la 
tha so-called "approach to a ataad y state," wharain tea haaard rate is constant 
regardless of tha failam pattern uf individual parte, This state ooaam aa a 
raauit of tha nixing of parte of di ffomwt apes whan failad alananta in tea aya* 
ten am replaced or mpalmd. Aa sxaap*s, unaa that a ayataa ooetalna parts 
which hara increasing haaard rates, When all parts are new, tea system haaard 
rate in low j it i:.. re anas aa tha parte age. When a failad part la replaced by 
a now ona. tea system haaard rate decrease*, and It falls sharply whan a master 
of replacements occur. However, it will Spain start to riaa an thana "second 
generation" porta bapin to aps- Thus, over a pariod of time, tha aya tan haaard 
rate decreases, uevJ it falls sharpy whan a master e* raplaoana n ta occur. How- 
ever, it will again start to riaa «s teas# "second generation" parts bapin to 
apa. Thun, ovwr a pariod of tim, tea ayatan haaard rate oscillates, but this 
cyclic so w an t dlainiabaa in tlaa and approaehaa a atsbla atata with a conatant 
haaard rate. 

* third Justification for assuming tea ax pona n tl^ distribution is teat tha 
•xpcsantlal is uaad as an approximation of aosss other density osar a particular 
interval of tisn for which tha true haaard rata la fairly constant. For example, 
if the true hat art- rate function is aa shown in the cur** below {assume that tbs 
ayatan la dsbujjpd), asvtnptian of on exponential for tha period from 0 to 250 
hours will piva a reasonable approximation. 
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These argue i nte notwlthatand lng, indiscriminate us# of the exponential (or 
us# of SBH distribution, for that ssttor) esn lead only to con- 
fusion and Uwonrctuu. It Is thsroferu obligatory on tha part of the analyst 
to validate tbs us# of any particular distribution function, Broadly spooking, 
tbsrs art two appmschss to validation; (l) Historical — i.e., exaalnation of 
tbs past psrfomanns, ubsrs available, of tha item; (2) Statistical — 1.#., 

"f auadnsss of fit* (Jhl- square and Kolaogorov-auironv tests). 

*.♦,5.2 pi stribution 

tbs probability density of tbs tvo-paasctsr WalbuJ 1 dlstributloc la given 

f(t) - £ t^ 1 (38' 

•bars 

Naan - a 1 ^ r Q- ♦ 1^ 

Purianos - a (2/ft+I) [r( 2/6 ♦l)-r 2 (l/S+ 1>] 

Cbarsctarlstles 

Tha flexibility of this density Is one of its desirable characteristics, a is 
a seals pa r -use ter and p la a shape piraestsr. When P » 1, this distribution reduces 
tc the exponential . 

Bel lability Applications 

Tbs Velball distribution is receiving wide application as the failure pattern 
of saaloonduotor defines and uachaaical devices, and because of its flexibility. 

It la also being used to desorlbe failure patterns at the unit end equipment 
levels. Tbs has and rate is constant whan P - 1, is an increasing function of 
tlas when p > 1, and la a decreasing function when p < i . 

b.k.5.3 gesem Putrlbutloo 

The probability density* of toe geeea distribution is given by 

-t/b a-i 

f(t) . • ,taO, a s 0, b a 0 

(a-1) .'b* 


where 


Itean » - b 
Variance .s/b* 


For "a* not an integer, (a-1 j. - r(a) * f e~ y y*~* iy 
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Characteristics 

The critical parameter la "a", whl;h controls the shape of the curve; "b* 

Is a scale parameter that determines the abscissa scale (l.e., o hanging b merely 
narrows or broadens '*.«* curve). When a is equal to 1, the distribution reduces 
to the exponential. 

Reliability Applications 

The gamma distribution is Important in reliability for thro reasons, first, 
it is an extremely flexible distribution and can therefore be used to fit the 
failure pat ten; of items in their various stages of development. When a - 1, the 
hazard rate le constant. It Increase* with time when a is greater then one end 
decreases with time whan a is leas than one. 

The second reason is that the estimated mean life of the commonly used expo* 
nential distribution has a ganma density, which can be used to make probability 
statements for estimates aad tests of the mean life. 

4. 4. 5. 4 Normal Dist r ibution 

The probability density of the normal distribution is given by* 


f(t) 



— < t < 0, 


(to) 


whsre 


Mean » n 

2 

Variance * a 


Characteristics 


The normal Is one of the most widely used continuous densities. The density 
function is a symmetrical bell-shaped curve, as shown on the left. The cumulative 

function is not directly lntegrable, 
but tables of the normal cumulative 
distribution with a mean of zero and 
& variance of one (called the standard 
or normalized form) are widely avail- 
able. These tables can be used for 
any normal distribution by transferring 
the original t variable to a new vari- 
able, y, by the equation 



7 - 


Srli 

a 


( 41 ) 


*The letter t rill be generally used to denote the random varla’-'e, which is 
consistent with the use of this letter for failure time. 
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The variable y Is normally distributed with a mean of aero and a variance of one. 
Thus, to find the probability that t Is less than a ay, c, one can use the tables 

by first letting y ' .1L.ZJi *nd finding the probability that y a y ' from the 
tables, A condensed table of the standard normal is given in Appendix C. Under 
appropriate condition! the normal distribution can be used to approxomate the 
binomial and Poisson probability laws (see any stanaard statistical text). 

Kragole: Assume u - 100 and o - 5. It is desired to find 
the probability of obtaining a value between 95 and 110 on a 
single trial. 


Let 



Then 


p (95 < t < 110) - p [-1 < y < 2] 

- P(2) - ?(-i) 

- 0.977 - 0.159 


- 0.818 


Reliability Applications 

Frequently, the normal distribution applies to items in which the failure 
occurs as a result of some wear-out phenomenon, since the hazard rate of the nor- 
mal distribution increases with time, in a manner consistent with a wear-cut proc- 
ess. Since the normal distribution implies both negative and positive values, it 
should not be used unless one of the following three conditions 1 b met: 

(1) u/o £ 3 (this condition establishes that the probability of a negative 
failure time Is small enough to ignore) 

( 2 ) all negative times observed as zero-hour failures are the result of 
wear-out during production testing, checkouts, installations, etc. 
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(3) A truncated normal distribution is employed that distributes tbs 
probability area frogs -• to 0 ovsr tbs poaitive rants 0 to •. Tbs 
distribution's density is 


where 


f(t) - 


0, for t < 0 

Z7h* 


( 42 ) 



A characteristic of the nornal distribution is that the mean life and 
median life are each equal to the time interval for which the reliability is 0.50. 


Example: Assume that an item wit x a normal time-to- 
fai lure distribution has a mean life of 500 hours and a 
standard deviation of 40 hour* (p. - 300 > 3c • 120). What 
is the probability that this item will operate at least 
250 hours without failing? 


R(25G) 


If y - 

y - 


- 1-F(250) 



"W ~ , the limit i. ■ 250 transforme to 
250-300 , „ 

-nr — 1,Za 


and 


R(U250) 


R(y— 1.25) 



1 .-y 2 /2 dy 

75" ” 


- 1 - 0,106 
- 0.894 
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The probability density of the log-nonaal distribution Is given by 


“ STTSF •" 1/5 ( < l0 g l " 7 ) 2 *' 0 > (^3) 

where 

Mean - s'* + ® 2/2 

Median « e^; log (median) - y 

Variance « e 2 ^ + (*f° ->l) 


Characteristics 


If the logarithm of a variable is normally distributed, the variable has a 
log-normal prooablllty distribution. The probability density function is posi- 
tively skewed, a large variance being associated with much skewness. Three log- 
normal distributions with identical 
means but different variances are 
plotted at the left. 


Reliability Applications 

The hazard rate of the log-normal 
Increases with time until the mode 
(most likely failure time) Is reached 
(the time corresponding tc the maximum 
ordinate of the density function), 
after which it decreases. The median 
life Is the more convenient and usual 
measure of central tendency since ~ 
unllke the mean — It Is Independent 
of the variance. 



Thi use of the log-normal distribution has been found to reflect adequately 
the fallu-'e pattern of many semiconductor devices and is also often appropriate 
for system or equipment tlme-to-repair distributions. 


4. 4. 5. 6 Other Distributions 


There 
discussed . 


are many other Important probability distributions 
Such distributions as the (chi square) and the 


that have not been 
t are often used 
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for inferential purposes. Son* of th« specific bmi of these distribution* la 
reliability and maintainability analysis are dlscut vd la tbs followir •’•otlon. 

4.S STATISTKM. MOTHS If TUT lift - UDMTNK 

This sootlon reviews luipartAnt statlstloal aspects of the evaluation of 
tests conducted to make reliability and maintainability inferences about a popu- 
lation of Items through estimation procedures. The purpose of estimation is to 
describe pertinent characteristics about & population through analysis of data 
o.i samples. The two major approaches to estimation are as follows; 

Nonpar aaetrlc sstlmatea . those which are made without assumption of any 

particular form for the probability distribution. 

Parametric estimates, those which are based on a known or assumed distri- 
bution of the population characteristic of Interact. The constants in the 

equation that describe the probability distribution are called parameters. 

As an example of these two approaches, suppose it JLe desired to estimate 
the prooablllty that an Item will survive for 50 hours. Twenty sample items ere 
tested until tl*7 all fail. For the parametric estimate of the 50-hour survival 
probability, R(50), if an exponential distribution Is aasiawd, R(50) can be 

A 

obtained from the expression e"^ 0 ^’ where 0 is ths estimate of the mean tlma to 
failure based on an exponential distribution of failure times, "'r ths nan- 
parametric estimate, the estimate of R(50) Is simply the propor . of the staple 
that survived 50 hours. 

Generally, nonpar ametrlc estimates are not as efficient as parametric esti- 
mates, since the fonror require greater sample slsss to achieve the sen precision 
as the latter. On the other hand, since no assumption about ths population dis- 
tribution Is made for nonpar am* trie teats, errors arising from Incorrect assump- 
tions are not encountered. 

The three cannon types of estimates are: 

(1) Point estimate — a a Ingle-value estimate of a parameter or charac- 
teristic 

( 2 ) Interval estimate — an estimate of an Interval that la believed to 
contain the true value of the parameter or characteristic 

(3) Distribution estimate — an estimate of the probability distribution 
of a characteristic 

The most common type of point estimate la the maximum-likelihood estimate, 
l.o. , the value that has the maximum probability of producing the observed sample 
results. A confidence-interval estimate , the moat common type of interval esti- 
mate, Is one for which there is a known degree of confidence (in a probability 
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b«m) that the true value of the unknown p a ga n tor or characteristic lias within 
a computed Interval. Whenever possible, a confldsnos -interval estimate should bs 
given along with the point estimate, for than the degree of precision In the point 
estimate oan be assessed. Tor example, assume that a 100-hour MFEP is desired. 
Sallee of two different designs are tested and the results are as follows: 

fflEOttfiEdltta ISflU Item B 

A 

Point Estimate, 6 95 hours 105 hours 

906 Confidante Interval ( « L , * v ) (90-115) (40-170) 

Although the point estimate for Iteci B lu above the 100-hour requimwnt, It 
is seen that the precision of toe estimate as date mined frees the length of the 
oonfldence Interval Is poor In comparison with tjhat of Item A. In this ease, 
since it is more certain that Item A will be close to or exceed the requirement 
than It Is that Item B will, the former may bo chosen. If only the point esti- 
mates were considered, the reverse decision probably would be made. 

* 

Two steps are generally Involved In making a distribution estimate: (l) 
hypothesising or determining through data analysis the form of the distribution, 
and (2) making point estimates of appropriate parameters that will completely 
describe thi distribution. 

Sections 4.5.1 and 4.5.2 summarise various types of estimation procedures. The 
general approach for analyzing test data for estimation purposes consists of the 
following steps: 

• State objectives for test data analysis 

• Determine appropriate forms of statistical estimates to meet objec elves 

• Perform any neoeeaary preliminary analyses such as analysis of the dis- 
tributional form 

» Determine If parametric or nonpar ans trie procedures are to be used 

• Apply appropriate procedures or equations to obtain estimates 

• Note unusual data results and set up test plan for confirming any new 
hypotheses 

• Report on results completely, describing test design, data collection, 

raw data, and data analysis 

4.5.1 Nonpar sme trie Estimation 

K sunsMry of various nonparsmetrlc estimates l* presented In this section. 
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*.5.1.1 Point and Intanral of ItolUblUty or Maintainability 

Tha almplaat aatlmata of roll ability for a tlaa interval (t), dmnoted by 
R(t), la to oaloulate tha proportion of lt«M that aunrlve ovor that tlao lntmr- 
val. Thus If n Item art put on teat, and f failuraa occur bofora tlao t. 


*{t) - fl=t 


m 


Similarly, tha probability of ocaplatlac a spoclflod aalntonanoo action by tlao 
t la 


g<«) . { 


<*M 


wbaro r la tha uuabor of auoh aotlona collated by tlao t out of a total of n 
rap air aotlona. 

Thaao aquation* art for tha oaaa of no withdrawal of ltoatt (oonaorahlp) 
during tha taat. 

Conatructlon of a confidanoa interval about R(t) or A( t) la baaod on tha 
faot that thaaa aatlaataa corraapond to a binomial parana tor. Tha aquatlano for 
oonfldanoo 11*1 ta ara aa follows t 


Lowor (1 - a)f Unit* 


^" 0 “ [ X * '<» + 2, 2n - 2f )] 


(t6) 


wharo 


A oondanaad sot of 


1^ (2f + 2 , 2n - 2f ) la tha uppor agf point of ♦‘ha f distribution 
with Jf +2, and 2n - 2r dagraea of fraado 
9 valuas la p.maantad in Appsndlx C. 

Uppar (1 - a Limit 


V 1-a 


1 + - : -TT T. ■T" W- 'g K ~ g t gf -t- g) 


(♦7) 


For a two-aldad (1 - a)% limit, tha Interval la 

^ *L,o/t>, 1 - o/2 ) 


In this appendix, (1-a) % la «o ba intarpretad as tha (1-a) fractlis or, 
equivalently, aa 100 (1-a) Jf, vhara a i. a daclmal. 


(< 8 > 


A-30 


t - - rot-lit 

o *»»•*>. 

,.l f prM ~ 41a » "rll.tUj to at apr^rU* “ *»*• 


"" 90f-floofid«w# uhi' 

(1) *«■ Iqa*tion **, 

<(60) . 5^22 . 0<80 

(2) f*a» Jqu*tion D6, 

*L, 0.10“ V[l + ■§ » 0<10 (», go) J 

*«* ?4blc C-4, 

*0.10 (22,60) . 1.5 


*L,0.10 * l /[l ♦ $ ( 1.5 )] 


*0.71 


rr-r 1 ' ,or * • °- a - “» 

yl * W * * mu * <* H’PnxlnAUXy 0 .«*. 

(3) Jr** Equation 47 , 

^0*05 " 1 / l l + 'o.OJ, <».*)] . 0.680 

Vo.M •»/[>.# - 0.888 

** — “ « ~ u 


*ntr«t#d from RADC R.im Xlity 


Notebook. 
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Tha aquations ara uNd in tha mm way for ©onf JLdanoa Units if 
tide data rathsv than fallura -tins data ara bain* analysad. 

4.5.1.? Sal lability functlona 

It point aatinataa ot R(t) art nada for rarinos raluaa of ft, a ralatlo n ahlr 
of R to tJ \m, ft, which la tha reliability function, oan ba dawalcpad. Tha ra li- 
ability eaaa la dlacuaaad In data il aara alnoa aaa an tlali y tha eaM prooaduraa 
ara uaad for nalntalnsblllhy. 


Mo Caaaorahlp 

Two iwthona ara poMibla whan no 1 
withdrawn for raaaona ofthar than fallura. 


ara oansorod — i.,a„ no it 


(1) Estimation at flaad points in tlna, tji 


whara 


' l 


R(t A ) - 


JZ1 . 


(♦9) 


whara 


n » nunbar of ltaaa orl*lnally on ftaat 
f j • nunbar of fatluraa In tha into nr al t ^ < t a tj 

F(tj) » nwibar of falluraa occurrin* on or bafora t^ 

(?) Eatiaatlon of Rft ) at fallura tlaaa, t^s 

"<V - R wH J 

k la tha nunbar of falluraa occurring on or bafora tha ordarad 


(50) 


fallura tlaa t.a. 
t l 4 *2 * *3 4 


tha fallura ttnaa art ordarad ao that 


thua If tha fourth fallura out of 10 obaarvatlona ©crura at ?0 hours, 
tv - 20 VkJ ft(?0) - H"* • 7/11 

for larts* aanpla* (aay, n > 30), Ecu-*’-* 4$ and 50 ylald naarly ldantical ratulta. 
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SaagBlai Ib • fast of $0 1— , failure* occurred *t the 
fellovuc Sup— 4 !mni 7# l8f 25, 27, *5, 4l. 47, 50, 54,60, 
Obtain tto* ob — — reliability function at (l) «wry 10- Norn* 
period op to 60 hour*, and (2> each failure tine. 

(1) Ob—rred reliability function* at 10-Jour interval*, 
froa Squat loo 491 


*1 

h 

r(t x ) 

a-Ktj) 

Mt x ) 

n 

0 

0 

0 

5C 


1.00 

10 

1 

l 

49 


0.96 

20 

1 

2 

46 


0.96 

30 

2 

4 

46 


0.92 

40 

1 

5 

45 


0.90 

50 

3 

b 

42 


0.84 

6o 

2 

10 

40 


0.00 

Ob— reed reliability function* at 
fra* equation (50) < 

each failure tise. 


K 


n-k+i 

*(\) * 

0 

0 

51 

l.CO 

1 

7 

50 

0.980 

4? 

18 

49 


3 

25 

48 

0.941 

4 

27 

47 

0.522 

5 

35 

46 

0.902 

6 

41 

45 

u.582 

7 

47 

44 

0.863 

8 

50 

43 

0.8^3 

9 

54 

43 

0.824 

LO 

60 

41 

0.8o4 


The two function* are plotted in Figure •*-?. 

Ceneorahlj? 

If terminated or centred oheervatlon* occur, and eenaorehip take* place at 
fixed tiaee, then 



>1 


*-33 


Reliability, R(t) 


MKP 


where it the nrntor of lt«M starting the J** latemli fj U the aabir of 
failure* la tb* interval tj). At t^ 1# th* end of tb* (j-l)*t ln*ml ; 

mm lt*M will to wctH > top e* Bj - b,„j - fj_j - *j_j (wj_j - amber of 
censored Iter* at 



riCMf 4-1 

SSSimi KUMtin l«RM 


XX otoervaiiam ar* tenalneted or 
oen mr ed at fiUmi an estimate of th* 
reliability at th* tic* of the l tt fall- 
or* i* 

1 

«(*!> • II i^r (5*> 

J-l J 

where Bj l* th* nuator of ltow 'tart- 
la* th* 3 th interval. IX «*.->• than on* 
failar* occur* at a given tiee, th* 
n«M rotor toooMQ - f ^ ♦ 1, wtorw 

X j la tto motor of falluraa oceurrlttg 
at tto 1 th failure tin*. 

for taminated or censored 
occurring readonly within a tin* ir. lar- 
val to tween failure*. 




1 

it 


a 

«T 


rV« 

- wj 4 


r 


(53) 


«Un 


• , la th* auntor of withdrawals during tto tlM lnt-rval 


IjmUl For tto data of th* M** 9 l* given above lr> *ho 
C*a»or»Mp , iifat vtot ~rm good it** was withdrawn every t*n 
hour*. Calculat* tto rellaellity function at ten- hour 
Interval*. 


tto following value* ar* derived fro* Equation lli 


« 

J 

Interval 
( hour* } 

h 

i 


2£il 

1 

t. 

*{t t ) 

i 

0 

3 10 

i 

j 

50 

0.980 


10 

0.960 

* 

tG 

* 20 

i 

i 

46 

0.979 

2 

K 

0-959 


JC 

« 30 

2 

A 

46 

0.956 

3 

30 

0.917 

5 

30 

* *0 

1 

1 

43 

0.977 

4 

to 

0.896 

5 

••C 

* 5- 

.3 

3 

41 

0.9C7 

5 

5C 

0.831 

6 

50 

a 60 

2 

- 

jr 

0.946 

6 

60 

0.786 
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|M||i to tMiOffi to tha ratio— ti— *1— « abo— m tha 
own of "■» Ct—oralUp", vltMnwU mn sate at tha 
i»VWrtH ti— » 2 * 5 . 20 , 22 , 25 * * 7 . and 56 boors. Calculate 
•to roll ability fanatic® at tto falla— tin—. 

fto foil— — to— a— tori — 4 froa Equation 53 * 


i 

tofeor— 1 
lhanii 


!i 


W* 

Bj -^i 


4(t t ? 

1 

0 « T 

i 

2 

50 

49 

0.980 

7 

0.980 

2 

7 • 28 

1 

0 

*7 

47 

0.97s 

is 

0.959 

3 

18 • 29 

1 

3 

46 

44.5 

0.978 

25 

0.9>8 

* 

25 « 27 

1 

0 

42 

42 

0.977 

27 

0.916 

5 

27 « 35 

1 

0 

41 

41 

0.976 

35 

0.894 

6 

35 • 41 

1 

c 

40 

40 

0.976 

41 

0.873 

7 

%1 « *7 

1 

1 

39 

38.5 

Q .975 

47 

O.851 

8 

*7 • 50 

1 

0 

37 

37 

0.974 

50 

0.829 

9 

50 a M 

1 

0 

36 

36 

0 973 

54 

0.806 

10 

5* • 80 

1 

i 

» 

34.5 

0.972 

60 

0.78b 

b.3.1.3 


Mae* 






ton —lata* —hi 11 ty la o— ad, e— or a * Op — ly p— aa n ta a prol- 
ate— — wr a fel — a— — uaUy ba eop«^-_#d until all —tot—— action* a— 

• —plate, fto ncapa— ferlc — tteaUa of tho probability that a aalntananca 
sitl— will — aopplatetf by ttea t ara exactly ua e— plaannt or tha —liability 
fur—1— If f, feta n— bar of falla— a, is —pieced by r, tha number of caaplatad 
r— air — felo—. 

Thua, for — feteatlap H(t) at fixed points i.i tl— tj, tha following la 
obtain— for tha iwaao—hip ea—i 

1 

*( <■*) - 1 V rj (5^' 

>i 


ate— 


a • nurbar of —lata— a action* obae rved 

tj • a— r of eoaplat— —pal— La tna inte.-raJ • < t * t, 

7*r as feteat— at —pai r tl— a, 

• <*»> * s4t 

whan k la tha a— bar of —p air action* so— let— on or befo— tha erde— 1 repair 
ti— 
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Example s Assume that nine repair action* are observed 
as follows: 0.2, 0.;, 0.4, 0.4, 0.5, 0.6, 0.8, 1.2, 2.0 


Then, 

from Equation 55 

, the fallowing la ■ 

obtained: 

t 

£ 

&Lbi 






1.0 

r 

0.2 

1 

0.->0 


— 

0.3 

2 

0.20 

0.6 

* 

0.4 

4 

0.40 



0.5 

5 

0.50 

* 0,2 

. f 

0.6 

6 

0.60 


/ 

0.8 

7 

0.?0 



1.2 

8 

0.80 

0 

0.4 0.8 1.2 1.6 2,0 

t 

2.0 

9 

0.9*? 



1.4 C 

onfldence Limits 

for Reliability and Maintainability Functions 


Estluavlng reliability or maintainability at fixed points in time without 
censorship corresponds to estimating a binomial parameter. Equations 46. 47, and 
48 can be used to obtain limits for the observed functions for cases of no censor- 
ship . If censorship takes place, the number of sample Items varies. Zf the total 
number of censored items, w. Is small compared with n (say w/n < 0.10), then, as a 
rough approximation, the sample-size value to be used is n-w/2. 

4 . Measures of Central Tendency and Dispersion 

The usual measure of cencr&l tendency is the mean or average value; and the 
measure of diaper sJ.cn Is the variance .r ifca square root, the standard deviation. 
For the nonparamet ric caae, these measures are valid only If the data are not 
truncated or censored — that Is, for the reliability ease all sample lcems are 
tested to failure, and for the maintainability caae all started repair actions 
ere completed. 

If t. represents cither a failure time or a repair time, the mean or average 

-a 

value Is estimated by 


n 



1-1 


where n Is the number of observed times. The variance is estimated by 

•* - tAt l w> 

1-1 
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For large a (ssjr great er than 30), the central-limit theorem may he used; 
this states that the quantity 

Y n * (58) 

has approximately a normal distribution with mean 0 and variance 1 where ii and a 
the population mean and variance, respectively. 

For large n, s can do used as an estimate for a, and an approximate (1 - a)% 
two-sided confidence interval for u Is obtained from the equation 

*[& ‘ *0/2, n-l 8/ ^ * * • * + *0/2, n-l 8/ *“ ] ^ 9) 

where 

*o/2 n-1 is th * a//2 percentage point of the t statistic with n-1 degrees 
of freedom. These values are tabulated In Table C-2. A one-sided limit 
is obtained by replacing t Q ^ R _ 1 by t Q for the limit desired. 

Example: Aseume that 30 failure times are observed and that 
T - 1^6, s - 40. Then the lower 955* confidence limit for the 
mean failure time, 8 , Is 

P [*"*0.05, 29 s a 
or 



The estimate of t Q is obtained by constructing the reliability or maintaina- 
bility function by the methods described and by plotting the distribution to find 
the value of t for which R(t) or M(t) » 0.50. For the reliability case, this pro 
cedure requires that testing continue until at least naif of the items fell. 
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Confidence intervals for iw obtained from tbs e«juatien 

n-r n 

p [*? < *0.50 < t n-r+l] " Z i!(n-ij! (£) ^ 

i«r 

where t p and - n . pfl are the r th and (n-rfl)^ observed ordered tines in the 
sample. Note that the confidence levels that can be used ire restricted to the 
values obtainable from the right-hand side of Equation SO. 

One-sided limits are given as follows: 



where 



n.' 


i!(n-iJT 


n(n-l)(n-2)**«{n-i-l) 
" — 


Example : Prom the data of the example given in Subsection 4. 5. 1.3, 
the median repair time t Q - 0.5. Prom Equation 60, 

6 

P [ t 3 < t 0.50 < *7] “ Z i-'ll-iJ"! (?) “ 0,0203 

i”3 


Tableo of the binomial distribution can be used to evaluate the sum on the 
right to yield a 90.8256 two-alded interval of (0.4, 0.8). For a one-sided upper 
interval, from Equation 62, 


P 


*0.50 < 




-0.9805 

or a 98.0556 upper limit for t Q ^ - 1 . 2 . 
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P-Fercent Ranee; A Measure of Dispersion 

?he 50 # or interquatll# rang* defined by 

t 0.50 " *0.75 ' *0.25 ^ 

la often used to muun dispersion In ncnparaaetrlc estimation pioeedures. For 
reliability ty la the value o f t for which $( t) - 1-P, while for maintainability 
It la the value of t for which M(t) • P. ^ is the number of hours over which 
the middle 50# of the sample observations were recorded. For the data of Example 5, 
- l.o - 0.325 - 0.675. 

Values of P other than 50% can be used. For example, the 90-percent range 
T 0 90 " ^ 95 *0 05 * or the reliability case, with truncated (non-f ailed) items, 

the P -percent range can be used if only the minimum value of R(t) le less than 
0.50 and the maximum value of P is [1-2 min ft(t)]. 

4.5.2 Parametric Estimates 

Statistical estimation procedures based on a known or assumed form of the 
probability distribution function are presented in this section. The character- 
istics of the distributions considered hare are discussed in Section 4.4. 

4. 5. 2.1 Determining the Form of the Distribution 

The validity of parametric estimrtes depends greatly on the validity of the 
assumed distributional form. In some cases, the knowledge of the experiment that 
produced the data will dictate what the dlatrlbutlon should be. For example, in 
testing for defects, tha number of defective items in a sample of n items la dis- 
tributed blnomlally if each sample item la randomly and Independently selected 
from a lot and tested, and if the outcome is either good or defective. In moat 
cases, however, there Is no Indication of what the true population distribution 
Is. Two fairly simple procedures for analysing tent data to determine the dis- 
tributional form are presented below. These procedures are called goodnesa-of-fit 
tests. 


Graphical Procedures 

The graphical procedures for goodness of fit Involve plotting the aample 
distribution and comparing it visually with the generic forms of known distribu- 
tion functions. To add In such types of analysis, special graph papers have been 
constructed so that when the observed distribution Is plotted, a straight line 
will result If the distribution conforms. 

T. test for the exponential distribution, wliare R(t) - it is noted 

that In :i(t) ■* -t/6. Thus If "ie observed reliability data conform to the expo- 
nential failure law, the natural logarithm of the observed reliability function 
will plot as a straight line against t. 
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Special types of graph paper for the. normal, log-normal, and Weibull 
distribution:, can be used for goodne»s-of-fit teste. 

Kolmogorov-Smlrnov Test 

The Kolmogorov-Smlrnov test is an analytical procedure for testing goodness 
of fit, although the easiest means for performing such a test is graphical. The 
procedure Involves comparing the observed distribution with a completely speolfled 
theoretical distribution and finding ths maximum deviation. This deviation xs 
then compared with a critical value that is dependant on a preselected level of 
significance. 

The steps are aa follows: 

(1) Completely specify the theoretical distribution; that is, if the 
distribution to bs tested has k parameters in the density function, 
values of each of the k parameters must be specified.* 

(2 ) Obtain the observed reliability or maintainability function and plot 
on a graph. 

(3) Find the critical value d from Table C-5, Appendix C, for the selected 
significance level and observed number of failures. A significance 
level of a me^s that or# of the time the test will reject the hypothesis 
that the distribution conforms to the one under test when in fact it 
does. Often this is stated aa the 100 (l-a)ji confidence level. 

(4) Draw curves at a dlstanoo of d above and below the speolfled theoretical 
distribution. These curves then make up a decision band. 

(5) On the same graph, plot the observed distribution. 

(6) If the observed distribution falls completely within the deoision band, 
the conclusion is thac the resumed distribution is correct. If any 
point of the observed function falls outside the deoision band, the 
assumed distribution is rejected. 

In many cases, one is Interested only in the form of the distribution and 
has no basis for parameter specification. In those oases, the parameters can 
be estimated from the test data to obtain the theoretical cumulative function. 
However, the critical d valueo in Table C-5 are too large an? will lead to 
conservative resulte (lower significance level) since xf the observed d value le 
greater than the critical d value, there is high, assurance that the hypothesized 
population form is incorrect. However, the chances of acceptijig the hypothesis 
if it le false is also increased. Results of Monte Carlo investigations have 
shown that the following adjustments to Table C-5 oan be made to yield approx- 
imately valued critical values for the normal and exponential distributions. 

Normal distribution - estimate u 4 c from -Us data 
Multiply d values in Table C-5 by 0.67 

♦See following discussion for the case in which parameters are estimated frem 
uiie dita • 
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Exponential Distribution - estimate $ from the data 
Multiply d values in Table C-5 by 0 80 

tx eanle i Aseuue that the following failure times are 
observed when a total of 20 items are tested: 

18, 25, 28, 39, 40, 48, 60, 66, 80, 8l, 83, 96, 105, 

108, 130 (5 Items survived past 130 hours) 

(1) Obtain Theoretical Distribution 

Suppose a test is being made for an exponential 
distribution. Slnco the reliability function for the 
exponential Is R(t) ■ e where 9 Is the mean 
failure time, the following estimate can be used: 

a _ Total Observed Life* 

Humber of failures 

m Total Time for Failed Items + Total Time for Non-Palled Items 

Number of Failures 

0 m 8§ g ^5 2 - 107,5 hours 

Then the estimated theoretical reliability function Is 

;< t ) . e - tA07 - 5 

This function Is also plotted in Flgnr. 4-3. 

(2) Calculate Observed-Reliability Fimctlon 

Equation 49 provides the following calculation for the 
observed reliability function (plotted In Figure 4-3): 



k 

n-k+1 

t >/a. \ n-k.+l 

R( V “ -n+r 

8 

1 

20 

0.952 

10 

2 

19 

0.905 

18 

3 

18 

0.857 

19 

4 

17 

c.eio 

40 

5 

16 

0.762 

48 

6 

15 

0.714 

60 

7 

14 

0.667 

66 

8 

13 

0.619 

80 

9 

12 

0.571 

81 

10 

11 

C.523 

83 

ll 

1C 

0.4?* 

96 

12 

9 

0.428 

105 

13 

8 

0.380 

108 

14 

7 

0.333 

130 

15 

6 

0.285 


•See Subsection 4 . 5. 2. 3, 


R(t ) 
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(3) Obtain Critical d Value 

Prom Table C-5, Appendix C. the unadjusted critical d 
value for a sample size of 20 is 0.304 when testing Is being 
done at the 10# significance level. By use of the oorreotion 
factor 0.80 (since $ is estimated from the data) tha adjusted 
critical value becomes 0.243. 

(4) Plot Decision Curve 

The unadjusted deolelon curves are constructed by adding 
nd subtracting 0,304 to the theoretical curve, and the 
adjusted curves are obtained by addi " % 0.243 to the theoretical 
curve. Thtae curves are also shown in Figure 4-3. 
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(5) aautea 

Binoa the observed reliability function falls within 
tha dsslslon curves, ths hypothesis of exponentlallty 
oannot bo rejeoted. For small sample also*, tho doolslon 
curves aro quits wide. For this example, It Is llksly that 
other distributions, such as the normal or Weibull, will 
also not be rebooted. 

4. 5. 2. 2 Discrete Distributions 

The two most cannon discrete distributions involved In reliability and 
maintainability testing are the binomial and Poisson distributions. 

Binomial 

The random variable, x, Is the number of occurrences of an attribute in n 
independent trials when the attribute is classified by either of two mutually 
exclusive categories. For reliability and maintainability, the attribute of 
Interest It normally a successful outcome, that Is, non-failure or satisfactory 
repair. 

The probability denalty function Is expressed aa follows: 

P(x; p, n) -(p)p x (l-p) n " x (64) 


whtre 


P(x; p, n) - probability of x oocurrenoaa In n trials whan 

the constant occurrence probability on one trial 
la p. 

For tha binomial distribution, the mean and varlanca are: 


where 


Mean: u » np 

U la tt'.e expected number of occurrences in n trials 

O 

Varlanca: c« np(l>p) 

Estimates of thaaa values are aa follows: 

A r 
p . L. 

* n 


t?h*rc 


(.n-r) £ 


r lr che number of observed occurrences in n trials 


(65) 

(654) 

( 66 ) 


ii M 
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Squat Ions 46, 47, and 45 art ussd to obtain confidence Halts on p, For 
n t 30, Plgure C-l, Appendix C, can be ussd. 

Bxsmfalei in a tost of 50 ltsns, 46 successes wars 
observed. What Is the point estimate of suoesss proba- 
bility and the associated 90Jt- confidence Halts t 

p - r/n ■ 46/50 - 0.92 


From Figure C-l, Appendix C, the 90jt-confldence interval is 

(0.83, 0.97) 


Polason Distributio n 

The random variable, x, is the number of occurrences of an attribute per 
unit segment (e.g., per unit time). If an item exhibits a constant failure rate, 
the number of failures in a fixed period of time is Folsson-dlstrlbuted if fail- 
ures are replaced as they occur. 

The probability density function la oxpressed as follows t 
■*7 

-mt x 

P(xj m, t) - (67) 


where 


P(x; m, t) - probability ol x occurrences in t segments if 
Poisson parameter is m 

ra - the mean number of occurrences per unit segment 

For the Polason distribution, the mean and variance are i 

M'-axu at 
Variance: mt 

The estimate ol the mean is at; follows: 

m » ^ (68) 


where 


r is the number uf observed occurrences in t unit segments 

To find (1 - u)< confidence limits on m, given r occurrences lr. t unit 
segments, nw and must be solved for the following equations: 
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for • lower (1 - a)fl Halt, solve for M^ In the equation 


AMCP 706*ltt 


Than 


r-1 


L e-L^ 

k*o k! 


1 


u 


(69) 



for an upper (1 - a)$ limit, solve for Mg In tha aquation 



k*o 


Than 




for tvo-aldod Haiti, usa 1 - a/2 and a/2 in Equations 69 and 70, reepee- 
tivaly. Tablaa of tha Polaaon function art available for euch calculations.* 


Kxmglei Aaauai that tan sonatant- failure -rota ltaai 
ara put on test, aach for a pari c-3 of IX hour*. Whan they 
fall, thay ara replaced by naw ltaaa. A total of 13 fall- 
urai occurred. Obtain the estimate of a, tha naan n>iabar 
of fall urea par lX-hour lntarval, and obtain the 95 %• 
confidence lldti. 


Tha eitlaata of a la r/i. Sir,.* '5 failure! have been 
observed In ten 100-hour Into •vals, 


* r ’ S 

* • t " iff ■ a - 5 


Kenco, 1.5 failures par 100 houra can be expected. 


?roo Equations 69 and 10 , tha <T'3f-c qnf idanca Interval for a la 


(0.6b a a < 2.b7) 


•ft. iyietck and 0. Walaa, Tablaa of the 
Order, U.S. Wa.al Ordnance Laboratory, 




4-45 


AMCP 706-19 


4. 5-2-3 Continuous Distribution* 

Descriptions of and estimation procedure* for continuous distributions are 
presented in this section. Specific distributions considered are the exponential, 
norasA, las normal, and Vetbull. 

fttpcnentlal Distribution 

The randan variable, t, la ths number of unit segments occurring before an 
event- In reliability, t represents hours or cycles, and ths event la its* fail- 
ure. In nalrtair ability, t can be weinbsnaaoe downtime, and the avent is ‘‘be 
cocrpl^tio-i of a maintenance action. It is assume 1 here that t represents hour* 
and the events represent failures. 

Ths probability distribution function is expressed as follows i 


fit; X) - )<*“** 


(71) 


where 

t a 0 and X is the ms an number of failures per unit time (per hour), 
cowmnly called the failure or hasard rate 

Since X Is equal to the reciprocal of the mean nurber of hours before a failure, 
the following can be wrltteni 


f(t, * ) * J e ' t/9 


(72) 


where 


$ « 1/X • mean failure ties 

For ths exponential distribution, the ■-ear., variance, and hasard rate are« 


0 

- 1A 

(73) 

c 2 

- 1A 2 

(A) 

h(t) 

- x, a constant 

(75) 


Mean failure time, 9 , la estimated by Procedure I or Frooedure II. 
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gOMflttl 11 until r failu~» oeeuri 

• * S wSar , ‘ o H tSr8 8 ? Kflu w 5 " r < 76 > 

To obtain 1, tha follcnrint grooadurat are uaad: 

Proaadura Ia i ftt'ltcaeant Taat ( fallura* rwpalr*d or raplacad) 

T - nt f ( n) 

vhara 

n •* non bar of itana on .eat 

th 

- tin# at which tha r ‘ failure occurred 
r 

Prooadura Ib i Nonrap laceasant Taat 

r 

T - Y t t ♦ (n-rjt p (7b) 

1*1 


whtca 


t, la tha tin* of 1 ' h -o.dar#d fallur* 


?roc«du^w C*naora<i lta*a iwithd""***! or It.ss of nor-fO.'ad ices*) 


Failurwi Ras-’*»-ad: T *» 


♦ (n-c)t. 


J-l 


*b*r* 


tla* of J Varda rad carac-rahlp 


* nyas'fir 


sj» - r » 


sftiort-; 


Pal Jurat Hot R*Pi*ca<J: ? * 


f *k' '' 


I 
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4. 5. 2. 3 Continuous Distributions 

Descriptions of and estimation procedures for continuous distributions are 
presented In this section. Specific distributions considered are the exponential, 
normal, log normal, and Weibull. 

Exponential Distribution 

The random variable, t, is the number of unit segments occurring before an 
event. In reliability, t represents hours or cycles, and the event Is item fail- 
ure. In maintainability, t can be maintenance downtime, and the event is the 
completion of a m-ilntenance action. It is assumed here that t represents houre 
and the events represent failures. 

The probability distribution function is expressed as follows: 


f<tj X) - Xe" 


where 

t a 0 and X Is the mean number of failures per unit time (per hour), 
commonly called the failure or hazard rate 

Since X le equal to the reciprocal of the mean number of houre before a failure, 
the following can be written: 

f (t, e ) . J e ' t/0 (72) 


where 


g ■ l/X ■mean failure time 


For the exponential distribution, the mean, variance, and hazard rate are: 

0 - l/X (73> 

a 2 - l/X 2 (74; 

h(t) - X, a constant (75] 


Mean failure time, 9, is estimated by Procedure I or Procedure II. 
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l! Teat until r failures occur: 


Total Life Observed 


To obtain T, the following procedures are used: 

Procedure la : Replacement Test (failures repaired or replaced) 


where 


T * nt„ 


n - number of items on test 


t r - time at which the r failure occurred 


>: Nonrep laceiuent Test 


T “ Z *1 + ( n " r ^ fc r 


where 


t^ is the time of i -ordered failure 
Procedure T c: Censored Items (withdrawal or loss of non-failed items) 


where 


Failures Replaced: T » ^ tj + (n-c)t x 


tj - time of J -ordered censorship 


c - number of censored Items 


Failures Not repla- 


ced: T - y t^ + ^ tj + (n-r-c)t r 
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grocedure II : Testing Terminated by Stopping Rule on Test Times 

If the test plan is sud that th<* test terminates after a specified 
number of test hour’s, t*, have accumulated, it is possible that no failures 
have been observed. Then Equation 76 cannot be used, since it implies that 
the estimated 9 is infinite. 


In general, for Procedure II testing, if the number of failures, r, 
is small (say r s 5), a better estimate of 0 can be obtained by the equation 


0 



(81) 


where 


T is calculated as in Procedure I except that t la now replaced by t*, 
the time at which testing is stopped. 


Example 1: Twenty items are placed on test. Testing 
continues until 10 failures are observed. Calculate the 
estimated mean life of the items as based on (l) a replace- 
ment test, with the 10th failure occurring after 80 hours; 

(2) a nonrepla cement test, with failures occurring at 10, 11, 
17, 25, 31» 46, 52, 65. 79, and IOC hours; (3) the same non- 
replacement test with 4 items censored: 2 at 30 hours, 1 at 
50 hours, and 1 at 60 hours. 

(l) Prom Equations 76 and 77, 


0 « 1 „ 2t r . 20 
r r 


- 160 hours 


(2) Prom Equations 76 and 78, 


y t t + (n-r)t r 


- 1 « i=l_ 


- *42 + MU201 m 3*4.2 hours 


(3) Prom Equations 76 and 80, 


9 


r c 

£ ti + £ tj + (n-r-c)t r 

Jfal 

r 


. 442 + 170 + 6(100) 

io 


■ 121.2 hours 
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Example 2 ; Twenty items are plaeed on test and the test is 
terminated a^ter 100 hours. Calculate the estimated mean life 
of the items based on (1) a replacement teati with 8 items 
failing before 100 hours; (2) a non-replacement test/ with 
failures occurring as in Example 1(2). 

(1) Prom Equations 76 and 77, 

9 - - 250 hours 

(2) Calculations are the same as for Example 1(2). 


Confide nee- Interval Estimates on 8. Two situations have to be considered 
for estimating co-.-Tldence lntarvals: one In which the test Is run until a pre- 
assigned number of failures (r*) occur, and one In which the test is stopped after 
a preassigned number of test hours (t*) are accumulated. The formula for the 

O 

confidence Interval employs the x (chi square) distribution. A short table of 
X*" values Is given in Appendix C. The general notation used is 


X 2 (P» <*) 


where p and d are two constants used to chocse tha correct value from the table. 


The quantity p is a function of th~ confidence coefficient; d, known as the 
degrees of freedom, is a function of the number of failures. Equations 82 and 83 
are for one-sided and two-aided ICO (l - a) percent confidence intervals, respec- 
tively. Por nonreplacement tests with a fixed truncation time, the limits are 
only approximate. These confidence limits on mean life are as follows: 


Confidence 

Fixed Number of 

Fixed Truncation'*’ 

Interval 

Failures, r* 

Time t* 


One-Sided 

( - , 2T - \ 

/ 2T 

(Lower Limit) 

(jC 2 (a,2r)’ ) 

V X (o,2r + 2) 


(82) 


Two-Sided Limits 


/ 2T 2T \ / 2T 2T 

Vx S (|,2r) X $ (l-f»2r)/ U (§»2r + 2) x -(l- | ,2r + 2) 


) (83) 


t For non- replacement tests, only one-sided intervals are possible 
when r - 0. Use 2n degrees of freedom for the lower limit if sll 
n items on test fail. 


Example 1 : Twenty items undergo a replacement test. 
Testing continues until ten failures are observed. The tenth 
failure occurs at 80 hours. Determine (l) the mean life of 
the items; and (2) the one-sided and two-sided 956 confidence 
intervals. 
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- ( 128.29, > ) 

(2) Prom Equation 83, 

( -a-y-22 , freest , 

'X (f'2r >2) x (l-^,2r) / \ 26.30 6 . 57 / 

- (114.83, 459.67) 
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T*bi« C-7, Appardlx C, extracted from the RADC Reliability 
notebook, presents the factor d) for one-sided and two- 

sided confidents limits, et six confidence la vela of each. 

Multiplying the appropriate factor oy the observed tote] life T 
gives a confidence limit about £. 

Sample-Sice Consideration. Since the length of the confidence Interval 
depends on the number of failures. It is possible to calculate the required number 
of failures to ensure -- with a specified confidence — that th.. estimate of 0 la 
within a specified percentege of the true mean time to failure. If a normal 
approximation to the X 2 distribution Is used In order to be (1 - a)# confident 

that 0 1 b within 6% of the true mean, 0, that Is * 6^ - 1 - a, the required 

number of failures, r*, Is 


rf m 



( 84 ) 


where Z 0 is the standardized normal deviate corresponding to the a)< point of the 
normal distribution. Z Q is tabulated in Table C-l, Appendix C. 

<*ice r* is determined, the approximate total teat time required can be esti- 
mated by the equation T* - r*8', where 8' is an initial estimate of e. 

Sjcsagle. How many failvree are required to give 9U -percent 
confidence that the estimate d ie within 20-percent of the true 
value? What will be the total test time if e m 100 hours? 

Pran Equation 84, for 90-percent confidence, Z . Z n . 

. _ . _ CL U . 1U 


6 , 

( 0 . 20 )‘ 

If 9' m 100 hours, then 

T* - r*0' - 67(100) - 6,700 hours 

Table C-6, Appendix C, presents values of r* for selected 
confidence and precision levels, 

Reliability Estimates, fifth To estimate the probability of survival 
for a time t, the estimates of 9 (Equation 2-76) can be used In t)m equation 


R(t) - e' t/S 


( 85 ) 
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Thi» estimate 1* biased (pessimistically if R(t} > 1/6 - 0 . 367 ), especially if 
r is small. An unbiased estimate is 


R(t) - (1-t/*)*" 1 , r > 1, t < T 


( 86 ) 


where r j.b ths number of observed failures in T total test hours 


Example: If 10 fail ves are observed in 1600 hours, 
calculate the reliability estimate for a 30-hour period 
(note that s - 160 hours) 


Fran Equation 85 , 


R(30) - e’ 30 ^ 60 - 0.829 


From Equation 86, 

R(30) - (l - 30 / 1600) 9 - 0.843 


Confidence Limits on R(t). The confidence limits on e can be used to 
obtain confidence lints on H(t) by the equation 


k 


t/s 


L,a/2 S R(t) * e 


>t/9 U,l-a/s1 - 1 

J 


(87) 


Por a one-sided lower limit, 


P JV^L.a t R(t)j - 1 - a (88) 

Example t For a mean life of 160 hours, (1) what 
is the probability of an item surviving 100 hours? 

(2) What are the two-sided 95% confidence limits on 
this probability? 

(l) From Equation S5 ( 

R( 100 ) - e" 100/ ® - e - 10 °/ 160 . o.535 


(2) From Equation 87 , th*. two-sided 95% confidence limits are 


/ 1UU 1UU \ 

fg-5J.55, 0-333.S5 1 ^ (o.3S4, 0.7*1). 
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Percentile Estimates. To estimate the time period, t , for which there 
1* a reliability of R# the estimate T R is 


9 In 


(89) 


The confidence limits on T R define a tolerance interval, since these limits 
permit the statement that there is 100 (1 - a) percent confidence that R percent 
or more of the items in t.ic population will survive T R or more time units. The 
100 (1 - a) percent corfidence limits on T R erfe given below: 


Confidence 

isSacai 


Fixed Number 
of Failures; r* 


Fixed Trur cation 
LtouJLl 


One-Sided 
(Lower Limit) 


( QJXUu&.>m\ 

*X 2 (o, St) 


{JLJjlA£L 

v 9r(a,2r + 2) ' 


(90) 


Two-Sided 

Limits 


V 2/a ov.1 «/, a o_\ 




gU a.lA. N i 

2771a o-\ ) 


2r + 2) /(l-S,2r) 


(91) 


Example: For a mean life of l60 hours, wnat la the 
estimated time period for which the reliability is 0.80? 
What are the 95-percent one- and two-sided confidence 
limits on T r ? 


Since 0 m 160 hours. Equation 89 yields T Q qq 
n S in ^ - 160 (0.2231M - 35.70 hours 


The 95*percent one-sided and two-sided confidence ?n T^, from 

Equations 90 and 91, are 


( 9 Ll SU>y $ p a M L , ,) - (22.73, .) 


and 


, umygf&i&y ( 20 . 90 , 7 u.* 5 > 

Normal Distribution 

The ..oru, distribution is one of the most widely used continuous deneitlea 
because ( 1 ' it approximates the distribution of many random variables and (2) 
the sample estimates tend to he normally distributed with increasing ssmple slat. 
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If an item Us a nonasl distribution of failure times, Us failure characteristic 
is consistent with a wsarout process. 

The non* si probability distribution function is expressed te follows t 


f(tJU»«) 



-«• X t t m 


(32) 


rta naan, variance, and hazard ate are expressed as follows: 
Mean * p. 

2 

Variance - o 

Hazi fiats (.increases with t) - 


I -mr 


TfSTr+rtTJT fc * » 


h(t) - 


«M_ t - u 

/Wo [ l - r(0] 6 > 


03) 


where 


«(t) - sze ' 1 ' 2 (£ s k) , r(t) - [i -e tSt' I 


2(t4t ) 2 i 1/z 

noZ 


The aean is estimated as shown in Cases I and II. 
Caee I - All tested items fail: 


and 


u - 




n t 


;z - 


I •‘*-(1*0 

l-i l-i 

n{n-l) 


( 94 ) 


(95) 


where 


t x is time to failure of the 1 th it* 


A-5* 
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£Mt & ” Vnawseted T-^Jt (f of r. lteaa fall). 7. era are two Mthods for 

estimating the Man tha f -apMcal m thod and the negreaaion -ethods 

Oraahloal *> Ihod (rAi > 1 / 2 ) 

(1) Calcultt* th* nooparaae t rl c reliability function, R(t), uelng tha 
aoet appropriate of the aquations (49 to S3). 

(£) Plot R(t) on nortaal probability paper and fit a straight line Yield- 
ing the estimated normal reliability function fty(t). 

Then 

u - t 0>50 (96) 

where 

fc 0 50 * of t for which fig(t) -0.50 

5 * *0.50 * *0.84 (9T) 

where 

*0.64 - of t fox* wtticH Rjg(tr^ • 0.84 

MBtuiaa fltttag 

Cl) Obtain fift^) by an epprog-iate nonpar aMtrlc equation there tj la the 
1 th failure tine. 

(2) for each failure tine, tj, find the ronud deviate f , oorreepoodlng 
to RCtj), uelng Table C-l. flft^) ocrreeponda to l-j^tj). Thus, for 
R< t ± ) - 0.971, t t - 1.9) for ft(t t ) - 0.50, X. • 0. 


Than 


&-<?! 

u * . ? , « oad 

D »U 

(96) 

a • 

b z -re 

(99) 

where 


r • nustbar of failures 


"* I *» *" Z l ! *• t V» 

1-1 1*1 1-1 

. - f X j 2 

1-1 


a -55 
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Example s Assume that failure data are generated as chowr. 
In Table Vi. 


TAILE 4-1 

•KtmilMK IF TINC 71 FMIK Ml CEKItEI Tilt 

FIUIWMC A MILNE IN A SAMPLE IF St ITEMS 

Hours to 
Failure, tj 

Number of Completed 
Observations, r. 

Number of Censored 
Observations at 

V w i 

1300 

1 

4 

1692 

1 

3 

2243 

1 

4 

2278 

1 

3 

2832 

1 

3 

2862 

1 

3 

2931 

1 

4 

3212 

1 

4 

3256 

1 

4 

3410 

1 

4 

3651 

1 

3 

Total 

2^-11 

39 


graphical Method 

The graphical method la shown in Figure 4-4: 

^ " fc 0.50 * 

° - *0.50 " t 0.84 " ^QOO-2740 - 1260 


Regression Method 

The calc Tatiana for p and a using Equations 98 and 99 are shown in Table 4-2. 
The two-elded 100 (l - a) percent confidence Interval on p is 

( ff - V2, r-1 x 7T’ » + \/2, P-1 x it’) ( 10 °) 

wnare t Q ^ 2 is *he n /\ 2 percentage point of tne t dlstrl ntlcn with r-1 degrees 

of freedom, nils interval Is only approximate for truncated tests. Values of t 
are given In Table C-2, Appendix C. For t > 30, Table C-l, Appendix C, of the 
standardized normal deviate, Z, can be used. 
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» 




\ 


TUlt 4 2 

MWIWTHMl nSCSHK m EtTIKATM TIS HU Ml fTtttUt 
KfMflN 9f I RMMftl KSTtMVTIM, UKI M NXMI 
•IStRTATMS MVHVM wmmim 


Hoar* So 
Failure, t^ 

r — — - - — " - - 

Observed Reliability 
Function, *(4^) 

Normal Deviate 
Corresponding to R(t.}, 

Z 1 

1300 

0.980 

-2,06 

1692 

0 

• 

vo 

& 

-1.73 

2 43 

0.935 

-1.51 

£278 

0.910 

-1.34 

2832 

0.883 

-1.19 

2862 

0.853 

-1,05 

2931 

0.819 

-0.91 

3212 

0.778 

-0.77 

3256 

0.726 

- 0.60 

3410 

0.653 

-0.39 

3651 

0.522 

-0.06 

* . ..r .. 

xt ± - 29,667 


- 11.61 


c » xt^ • 29,667 u » xa^ * -11.61 r ■■ 11 


d - Xtj^ - -27,062.8 e - l* 2 1 • 15 77 

&-*§=» - 3972 

b -re 

3 • Sfc££ - 1208 
to -re 


Example. From the data for the preceding example, calculate 
the 95-percent confidence Interval for 9. 

For r - 11, and a - 0.05, * o .0S5, 10 ” 2,23 
Then, from Equation 100 the _,5-v*rc*nt Interval la 

(3972 - 2.23 ^-p, 3972 + 2.23 ]Sj£) 

- (3161, 4783) 
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Given and a, the reliability function is obtained from the equation 


R(t) - J 


1 " -— e' y /2 dy - 1 - F,Z) 
Z v'Ztt 


where 


z - 7S e * !,2/2 '“ r 


Values for cumulative normal distribution F(Z) are given in Table C-l, Appendix C. 


Example ; The reliability function "or the failure data 
given in Table 4-1 is presented in Figure 4-5. 

For example, to obtain R(2000) 

„ 2000-3972 , 

z _ — - -1.63 


F(-1.63) - 0.072; R{2000) - 1-F(-1.63) - 0.928 


Bar Denotes 95# 
Confidence Interval 
Around Mean 


Observed Function (Non-Parametric) 

Theoretical Normal Function Based 
on u and a Sstimated from a 
Censored Sample 

A 

ix » 3972 Hours 
a - 1208 Hours 
r 11 Failures 



2000 3000 4000 

Time to Failure, t, in Hours 

FIMIE 4 8 

INfAIAKTIIC «• TKUfTIMl lEUMIlItt fllCTIIBS 
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Weibull Distribution 

Many random variables of failures and repairs can be described by the 
Weibull distribution, which, because of its three parameters, is quite flexible. 

The Weibull probability dlatributi on function is expressed as follows: 


f{t) - a yt a ,p, -y > o 


( 102 ) 


where 

a - scale parameter 
p - shape parameter 
y m location parameter 

The location parameter y represents tne minimum failure or repair time. 
Often It Is set equal to zero, and the density Is then 


f(t) - e- tP/ ° t . o. 


a, p > o 


( 103 ) 


If p ■ 1, the Weibull reduces to the exponential. If g is known, analysis 
may proceed exactly aa for tne exponents 1 except that all times t^ are replaced 
by the values t^. 


The mean, variance, and hazard rate for the Weibull distribution are as 
follows : 


Kean - u ^ y + r(l/p + 1) 


( 104 ) 


where 


r(l/p + 1) is the gamma functior, which for integer values of (1/p + 1) is 
equal to (l/a)j 


Variance - a ™ 


o 2 /P |r(2/p + l) - r 9 (i/p + i)J 


( 105 ) 


Hazard Rate - h(t) - £ t^" 1 


( 106 ) 


Hot#. 


h{i) deoreaaea with t if p < 1 
h(t) is constant if p - 1 
h(t) Increases wltn t if p > 1 
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Estimates of 7. P, and a. Analytical procedures are available for 
estimating the a, p, and 7 parameters of the Weibuli distribution from test data*, 
but they Involve fairly complex lnterative procedures. 

A relatively simple graphical procedure ia usually used to obtain estimates 
from Weibuli probability paper. A sample of such paper is shown in Figure 4-6. 

TWo sets of scales are used. Tne left scale, F(t), and bottom Beale, t, are for 
plotting the raw failure or repair data. The right scale, Y, and the upper seals, 
X, are called tne principal ordinate and principal abscissa scales and are used 
for obtaining the a and p estimates. Tne principal abscissa is that horizontal 
line for which X « 0 on the right scale, and the principal ordinate is that verti- 
cal line for which Y - 0 on the upper scale. 

The procedure is described below for the case in which r failures are observed 
out of a sample of n. 

(l) Compute tne failure probability function by the equation 

»<*1> - nTT (K>,) 


where 

^ is tne time of the 1 th failure 
n is the number of items originally on test 

(2) Plot P(tj) versus t on Weibuli probability paper and fit a smooth 
curve through the points 

(3) Estimate 7. If plots as a straight line, 7 - 0. If Fttj) plots 

as a curve, a constant value, k, is to be subtracted from t^ such that 
the plot of Fftj-k) versus (tj-k) is best fitted by a straight line. 

The initial value of k can be either the first failure time, t 1# or 
the t intercept of the curve. Several values of k may have to be tried 
before a reasonably linear plot of points is obtained. The estimate 

of 7 is then the value of k that .produces a linear fit. 

(4) Estimate p. The estimate of 0 is the slope of the fitted curve. It 
can be obtained directly from the equation 

0 (109) 

where Y 0 is the intercept with the principal ordinate (Y 0 < 0) and X Q is 
the intercept With the principal abscissa. 

•For example, D. Lloyd anu M. Lipow, Re? lability; Management Methods and 
Mathematics, Prentice Hall, 1962, pp. 177- lol. 
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(5) Estimate a. Tne intercept of the fitted line with the principal 

ordinate ie equal to -in a. Hence, if Y Q ie the intercept (which le 
negative). 


a - e‘ Y o 


(109) 


a can alao be obtained from the equation jn a * p in t # , where t* le 
the value of t for whioh F(t) -= 0.628. 


Y 0 or X Q , or both, may lie outside the graph: 

(l) Y Q outside the grapn 

0 can still be estimated by picking any two points on X, say X^ and 
Xg, and finding the corresponding Y's, e.g., Y 1 and Y^. Then 


g Y 2- Y l 


a is then estimated by the equation 


A 

a 



( 110 ) 


( 111 ) 


(2) X Q outside the graph 

Since a does not depend on X Q and p can be obtained by Equation 110, 
this case presents no difficulties. 

(3) X Q and Y 0 outalde the plot 

Multiply the t scale ty an appropriate power of 10, e.g., 10 1 , 10 _1 , 

.O 

10 , etc. The elope ie independent of the scale, and therefore p is 

estimated aa before. The estimate for a Is obtained by the equation 

a - io-e a' (112) 


where J la the scale faotor and a' Is the graphical estimate of a when 
the data are plotted on the basis of the t x 10-* scale. 


Another possibility is that the Velbnll plot appears as two Intersecting 
lines. For this esse, two seta of a, 6, and y estimates are made, one for each 
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linear portion. Tie estimated density Is then 


fft) 


P 1 (t--y 1 A -l e .(t-y 1 ) p ^/a 1 


( 113 ) 


1 - 1 for t s t* 

1-2 for t > t* 

and t* Is the time at which the two lines Intersect. 

Example ; Table 4-3 presents failure data (grouped) for 
germanium power transistors. Saventy-five transistors were 
put on test for 7000 hours and 44 failures were observed. 
Failures were noted every 250 hours for the first 1000 hours 
and every 1000 hours thereafter. Since the tsrpi; jxzs it 
large, the formula for F(t) can be slightly modified by using 
n in the denominator of Equation 107 rather than n + 1. 

g.tep.J.1], 

F(tj) Is calculated aa shown in Table 4-3- 


TAKE « 1 

KtlUH M«fl TIMSISTMS. UeaiKJTm 

PKCEBT EAliFK IS. SEUtffi TIM Rim »U 

Failure-Age 
( hours ) 

Failures 

Accumulative 
Fallur«*8/S ample 

Slxe of 75 

. . . 

Accumulative 
Percent failure 

250 

17 

17/75 

22.7 

500 

8 

25/75 

33.3 

750 

1 

26/75 

34.7 

1000 

1 

27/75 

36.0 

2000 

0 

27/75 

36.0 

3000 

5 

32/75 

42.7 

4000 

3 

35/75 

46.7 

5000 

4 

39/75 

52-0 

6000 

3 

42/75 

56,0 

7000 

2 

44 /75 

38.7 


asssJ&l 

Figure 4-7 snows the plot the data on Weibull probability paper. The 
t axle Is (multiplied by 10^ to »ceo«sodet* tne failure times. A straight 
line fits the data well. 
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ISk5S3Sw55! 


WfcP9*6(l (lOPi) 


MNM-1 

MINK PttCtMit FM NT40KM (STIUTIS If FtUKTttt 

usKtiTii win m ttwni Mumvnia 

ill 

Since the points are fitted by * atrai*ht line, y • 0. 

The intercept of the fitted line with the principal abaeiaaa, X Q , la 
approximately 2.85 ! and the principal ordinate intercept, T Q , la appro-xl- 
mate 1 y -O.85. Thua, fro* Equation 108, 


isaJ# 

The intercept of the fitted line with the principal ordinate ia approximately 
>0.85. Bence, fro* Equation 109, 

a* „ e -(-©*85) . g0.®5 . A 

and the unecaled eetlawU la, fro* Equation 112, 

a - io3(0-3°) ( 2 .>a) . 18.6 
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tatlaate of the Moan (u). The Been can be eetlaated by rep lac lrqj the 
tetlaetes for a, p, and 7 In Equation 104. Then 


U - 7 + a T(l/$ + 1 ) (114) 

A ahort table of r (X) Oim tu presented in Table C-9, Appendix C. The 
nlatlonehJ? r(Z + 1) - XT(X) oan bo \sed for X> 9. 

txaaplo : Iheom tbs data of the preceding exeaple, 
estimate 4 • 

fro* Equation 114 

t. ».**■» fo.x) 

m 18,900 r (4.53) 

Since 

r (4.33) - (3.33) (2.33) (1.33) T(1.33) - 9.216, 
then 


£ . (16,900) (9.218' 



tsihtiui «f«u mnatm utmw iftj. r« 
HNMrttl (IMiUUI 


- 157,000 iioure 

letbaate of the Reliability 
ganctl an. ?ras the estimates of 
P, and r , 

S(t) - U 15 ) 

For valntainabillty, the probatillty 
that a repair is completed before t 
hour* is 

ft(t) . 1 - e ~ ' ns ) 

feugsgls - The reliability function 
for the data- or Table *»-5 Is shorn in 
glfure e-fi. 
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Point pad Low:- Confidence Halt on B(t)« . The point estimate end 
lever oonfldenoe limit on R(t) for unknown o end fi, when *■ failures out of n ere 
observed, ere obtained ee follows: 


(1} Compute 


where 


r 

In tj - in t 

e * 



r 

s * I e i ‘i 

1-1 


t is the ties period of Interest 
t t le the 1 th f eJlure tine 

e^ end b^ ere constant* given by Johns end Lie be men,* Tible II. 


(117) 


( 118 ) 


(2) The estimated reliability Is 




fi(t) -e 



(119) 

(3) Table I of Johns and Llebermsn* 
gives the value of fi(t) T . . the (1 -a)f( 

lower confidence bound on R(t), obtained 
by entering the table with the calculated 
(2,.^ b ) value. 

Estimate of Haeard-Bate Function . 

Pros Squat lor. 107, the estimated hasard- 
rate fit: .lc*; le 

S(t) - .' A (120) 

o 

Example . The hazard -rat* function 
for the data of Table 4-3 la shown In 
Figure 4-9. 


- Z a/b 


mail 

•mu uiMim nMciw. *iu. m 


•M.V. Johns aid QJ, Llebermen, *te Exert Asymptotically Efficient Confidence 
bound for Reliability la the Ceee of the Veibull distribution", Tecfwwvtrlcs, 
Volume 8, busier 1, February l$ r 4>. 
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Log-normal Distribution 

A random variable whose logarithm Is normally distributed is said to have 
a log-normal distribution. This distribution frequently describes repair-time 
di stri'outions. 

The log-normal probability distribution function is expressed as follows; 

f(t; = o5tfe e " 1/2 t > 0 (121) 


The mean, median, variance, and hazard rate of the log-normal distribution 


are: 


Mean 


. _ g« + to /2 


Median: m = 6 V 


( 122 ) 


(123) 


The median is often used *s a central- tendency measure for the log-normal 
since It is independent of co. 

Variance : c 2 = + (s* - l) ( 124 ) 

Hazard Rate : The hazard rate of the log-normal increases until the mode 
('go la pg^^ed, then decreases. 

Es timates . The simplest procedure for estimating the reliability or 
maintainability function for r data points out of a sample of n is to employ 
log-normal probability paper. By fitting a straight line through the nonparametrlc 
function (Equations 4^ through 53), the foj lowing estimates are obtained: 


» = in t. 


0.50 


m *> t, 


0.50 


(125) 

( 12 *) 


wnere 


t Q ^ is the time for which R(t) or M(t) = 0.50 


Eor tu. 


“ in *0.50 " in tr \84(0.l6) 


( 127 ) 


nhere 


tg is the time for which R(t) -» 0.84 or M(t) = 0„l6 
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( 128 ) 


o 9 -2 / -2 \ 

a 2 - m 2 e w (e“ -l) 


(129) 


Confidence Limits on Median, Confidence limits on v 1 can be obtained 


from the equation 


" V2> >lf iV ‘ * + VS, P-1 


= 1 


(130) 


where t ,, , 18 the (a/2 )# point of the t statistic with (r-l) degrees of 

CL/ £ $ r-A 

freedom (Table C-2, Appendix C). This represents a confidence interval on the 
logarithm of the time for which reliability or maintainability is C.50. Then 
for the median, m * t Q 5Q , for a (1 -a)# confidence interval 


f p °L, a/2 g«u, l -cs/s'j 


) 


(131) 


where 

and Uy are lower and upper limits on u, respectively. 

Example : Forty-six malntenance-actior. times on an 
airborne communications receiver are shown in Table 4-4, 
along with the nonpararetric maintainability function. 

This function is plotted on log-normal probability paper 
in Figure 4-10. 

It is seen that a straight line fits the data points fairly well. The value 
of t Q = 1.95, and the value of t^ = 0.56. 

From Equations 125, 126, and 127, 
u * in ty = 0 668 

® * * 0.50 = 1<95 

= In t Q 5Q •• in t 0>l6 = 0.668 + 0.579 = 1.247 
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Then, from Equations 128 and 129, 

£ - 1.95 e°' 78 - 4.25 
S 2 - ( 1 . 95 ) 2 e 1,562 (e 1,562 -1 } * 68.06 

From Equation 130, 


V L, .025 


0.668 - 1.55 h * .308 


V U, .975 


0.668 + 1.96 - 1.028 


and the 95-percent confidence interval on m is, by Equation 131, 

(1.35, 2.80) 


TAIL! 4 4 

C9HPITATIBNS FSR MAWTAIIAIIIITT FBNCTIBR 

Observed 

Data 

Non- 

Par ametric 
Function 

Observed 

Data 

Non- 

Parametric 

Function 

t i 

r i 

K (t ± ) 

*i 

r i 

K (t ± ) 

0.2 

1 

0.021 

3-3 

2 

0.681 

0.3 

1 

0.043 

4.0 

2 

0.723 

0.5 

4 

0.128 

4.5 

1 

0.745 

{ 0.6 

2 

0.170 

4.7 

1 

0.766 

0.7 

3 

0.234 

5.0 

1 

0.787 

C.o 

2 

0.277 

5.4 

1 

0.808 

1.0 

4 

0.362 

5.5 

1 

0.030 

1.1 

1 

0.383 

7-0 

1 

0.851 

1.3 

1 

0.404 

7.5 

± 

0.872 

1.5 

4 

0.489 

8.8 

1 

0.894 

2,0 

2 

0.532 

9.0 

i 

0.915 

2.2 

D 

0.553 

10.3 

1 

0.936 

0.5 


0.574 

22,0 

1 

0.957 

2.7 

■M 

0-596 

24.5 

1 

0.979 

3.0 

2 

0.638 
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A FORTIORI ANALYSIS. An analysis deliberately made to favor 
an alternative system when comp .rod to a judgmental 
“best" system. If the "best" sytiten receives a favorable 
comparison under the weighted analysis, its position is 
strengthened. 

ABSCISSA: The horizontal distance fror the vertical axis 
of a graph, usually designated x. 

ACCEPTANCE SAMPLING: Inspection of ’ampins of incoming lots 
to determine acceptance or rejection of the lot. It is 
characterized by the staple size n and tho acceptance 
number c, or by the average outgoing quality limit. 

ACCRUAL ACCOUNTING: The recording and reporting of expenses 
as the operating transactions occur. This method, in 
contrast to obligations and disbursements, provides a 
realistic measurement of resources consumed in doing 
the work. 

ACCUMULATOR: The register and associated equipment in the 
arithmetic unit of the computer in which arithmetical 
and logical operations are performed. 

ACTIVE REPAIR TIME. The portion of the down time during 

which one or more technicians are working on the system 
to effect a repair. This time includes preparation 
time, fajlt location time, fault correction tine, and 
final chrck out time for the system. 

ADDRESS: An Identification, represented by a name, label 
or number, for a register or location in storage. 

Addresses are also a part of an instruction woxd along 
with commands, tags, and other symbols. 

ADMINISTRATIVE TIME: The portion of the down time not included 
under active reyair time and logistic time. 

ALGORITHM: An orderly, *tep-by-step procedure for performing 
a mathematical operation in a finite number of stepr. 

The 1040 form is an algorithm for computing personal 
income tax. 

ALLOCATION: (1) The distribution of available resources 

to the various activities which muse be performed in 
s-ch a way that total effectiveness will be optimized. 
Allocation is necessary when there are limitations on 
either the amount of resources available or on the way 
in which they can be expended such that each separate 
activity cannot be performed in the mos* - . effective way 
conceivable. (2) An authorization by c designated 
official of a department making funds available within 
a prescribed amount to an operating agency for the 
purpose of making allotments. 


'Wor'i-iuced hy per*i*8t0'~, of ‘h** U. 3. Arny M*n*ges«T.t S.hc.' . 
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ALLOTMENT! An authorization grantad by an operating agency 

to another office to incur obligations within a specified 
amount pursuant to an appropriation or othsr statutory 
provision and subject to specific procedural , bookkeeping, 
and reporting requirements. 

ALTERNATIVES : The means by which objectives can be attained. 
They need not be obvious substitutes for one another or 
perform the same specific function. Thus, to protect 
civilians against air attack, shelters, "shooting” 
defenses, and retaliatory striking power ire all 
alternatives . 

ANALOG COMPUTER: An electronic device that performs mathe- 
matical operations on numbers which are expressed as 
directly measurable quantities, generally voltages 
ano resistances. Analog computers are less accurate 
than digital computers, but they are more readily 
adaptable to changes in the data and structure of • 
problem. They are especially well suited to problems 
involving differential equations. 

ANALYSIS OF VARIANCE (ANOVA) : The basic idea of ANOVA is 
to express a measure of the total variability of a 
set of data as a sum of terms, each of which can be 
attributed to a specific source or causa of variation. 

A°PORTIONMENT: A distribution made by the Bureau of tha 

Budget of amounts available for obligation or expenditure 
in an appropriation or fund account into amounts available 
for specified time periods, activities, functions, 
projects, objects, or combinations thereof. The amounts 
so apportioned limit the obligations to be incurred or, 
when so specified, expenditures to be accrued. 

APPRAISAL ; Impartial analysis of information conducted 
at each responsible management and control level to 
measure the effectiveness and efficiency of the total 
process and determine preventive/corrective action. 

ARGUMENT: (1) An independent variable; e.g., in looking 

up a quantity in a table, the number or any of the 
numbers which identifies the location of the desired 
value; or in a mathematical function, the variable 
which when a certain value i» instituted for the 
value of the function is determined. (2) An operand 
m an operation on one or more variables. 
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ARTIFICIAL INTELLIGENCE t The study of computer and related 
techniques to supplement the intellectual capabilities 
of man* As man has invented and used tools to increase 
his physical powers, he now is beginning to use artificial 
intelligence to increase his mental powers. In a more 
restricted sense, the study of techniques for more 
effective use of digital computers by improving program- 
ming techniques. 

ASSEMBLER: A computer program which operates on symbolic 

input data to produce from such data machine instructions 
by carrying out sivsh functions as: translation oi‘ 
symbolic operation code a into computer operating instruc- 
tions; assigning locations in stcrage for successive 
instructions; or computation of absolute addresses from 
symbolic addresses. An assembler generally translates 
input symbolic codes into machine instructions item for 
item, and produces as output the same number of instruc- 
tions or constants which were defined in the input 
symbolic codes. 

AVAILABILITY: The probability that the system is operating 
satisfactorily at any point in time when used under 
stated conditions, where the total time considered 
includes operating time, active repair time, administrative 
time, and logistic time. 

AVERAGE OUTGOING QUALITY LIMIT: The average maximum fraction 
defective leaving an acceptance sampling plan. 

BALANCE OF INTERNATIONAL PAYMENTS: A Bysteir-atic record of 
the economic transactions of a country during a given 
period which involve a t-ans # er of currency between 
the country’s residents ar.d the residents of the rest 
of the world. 


BAYESIAN STATISTICS: Estimates of (prior) probability 
distributions, subsequently revised (posterior dis- 
tribution) to incorporate new data by means of Bayes 
equation: 


P (A jJ 3) * 


P(BjA 1 ) ?{A X ) 

F (B | A x ) P(A X ) + P(BjA 2 ) P(A 2 ) 


BERNOULLI PROCESS: A random process that yields an either-or 
outcome on each trial with known probability of occur- 
rence, and results from statistically incependent trials. 


A- 3 


AMCP 706-191 


BIAS: An imbalanced range of error such that the average 
error is not sero. 

3 I NARY: A characteristic, property, oz condition in which 

there are but two possible alternatives} e.g. , the binary 
number system using 2 as its base and using only the digits 
sere (0) and one (1). 

BINOMIAL DISTRIBUTION: The distribution of rriny two- valued 
processes such as heads and tails, or acceptable and 
unacceptable units. 

• x i—x 

Prob(x heads in n tosses)- -r P (1-P) 

m-xj *x. 


BIONICS: The application of knowledge gain ad from the analysis 
of living system) to the creation of hardware that will 
perform functions in a manner analogous to the more 
sophisticated functions oi the living system. 

BIT: A unit of information capacity of a storage device. 

BLACK BOX: An unknown and often unknowable mechanism cr 

system whose operation is judged solely by observation 
of its inputs ~.nd outputs. 

BOOLEAN ALGEBRA: A process of reasoning, or a deductive 

system of theorems using a symbolic logic, and dealing 
with classes, propositions, or on-off circuit elements. 

It employs symbols to represent operators such as AND, 

OR, NOT, EXCEPT, IF... THEN, etc., to pwrmit mathematical 
calculation. 

BRANCH: The selection of one of two or more possible paths 
in the flow of control based on some criterion. The 
instructions which mechanize this concept are sometimes 
called branch instructions; however, the terms "transfer 
of control" and "jump* are more widely usad. 

BREAK-EVEN POINT: In engineering-economic studies, the point 
at which two site •'tivei become equally economical by 
altering the value of one of the variables in a situation. 

BUDGET: A proposed plan by ar. organization for a given period 

of time refl ecting anticipated resources and their extiaatod 
expenditure in the pursuit of obivetives. 

BUILDING BLOCf COST: One kind of a rough. estimate of the cost 
of an alternative lor planning purposes. The estimate is 
not time-phased and does not provide for variations such 
as in the manning of thj unit or cost -quantity relationships. 
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CXHTAAL UNIT TKSOXKNi If the sample size It large (n>30), 
the sampling distribution of the mimi X, can b« 
approxima ted closely with a normal distribution. Further- 
more, this theorem also applies when n<30 provided that 
the distribution oi the population from which the samples 
are taken ran be approximated closely with a normal curve. 

CSHTAHTTfi The state of absolute confidence in which outcomes 
are sure and predestined. 

CSmiKJB FJUIHOl i The assumption that all conditions other 
than the ones specifically being analysed rasuiin constant 
or unchanged. 

CHI SQCAJtl TSJTi A statistical test for relatedness of two 
discrete variables, say height and weight of officers. 

CLSAKt To erase the contents of a storage device by replacing 
the contents with blanks, or zeroo. 

COMBINATIONS t Humber of possible arrangement* of n elements 
taken c at a tine if sequence is Ignored. 


(n-c) !c! 


CON Mm «HU»IUTY i The capability of two qualities or values 
to be measured by a meaningful relevant coamon index. 

For example, machine guns and riflaa ara comae nsur able 
either in dollar coat or in effectiveness, a.g., er;.;*fy 
casualties. However, machine guns and friendly casualties 
are not coneiensursble in terms of dollars. 

COMPILKN* A computer program more powerful than an asaemblar. 

In addition to its translating function which is ganerally 
the us* process as that used in an assembler, it is able 
to replace certain items of input with series of instruc- 
tions, usually called subroutines. Thrs, where an assembler 
trine late* itae for it« 2 , and produces ss output the same 
number of instructions or constants which were put into 
it. a compiler will do more than this. The program which 
results from compiling is a translated and expanded version 
of tl.e original. 

COMFVTBX t A device capable of accepting information, applying 
prescribed processes to the information, and supplying 
the results of these processes. 
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CONDITIONAL PROBABILITY t The probability that A will occur, 
given that B has occurred i P (A | B) . 

CONFIDENCE i Tha degree of trust or aaauranca placed in a 
given result. 

CONFIDENCE INTERVALS i A measure of effectiveness in testing, 
expressed in quantitative terms ; e.g., the value of a 
specific factor (variable) lies within a specified interval 
951 of the time. 

CONFIDENCE LEVEL t The probability that the true value of a 
parameter lies within a stated Interval. 

CONSOLEt A portion of the computer which may be used to 

control the machine manually, correct errors, determine 
the status of machine circuits, registers and counters, 
determine the contents of storage, and manually revise 
the contents of storage. 

CONSTANT DOLLARS! A statistical series is said to be expressed 
in 'constant dollars* when the effect of changes in the 
purchasing po*?er of the dollar has been removed. Usually 
the data are expressed in terms of some selected year 
or set of years. 

CONSUMER PRICE INDEX. A measure of the perlod-to period 

fluctuations ir. the prices of s quantitatively constant 
market basket of goods and aervlcaa selected aa rapre San- 
ta ti vs of a specific level of living. Hence, it can be 
thought of as the cost of maintaining e fixed acala 
of living. 

CONSTRAINT! A resource limitation, wnich aaj be specific 
(e.g., the aupply of skilled manpower or a particular 
metal), or general (e.g., total avallaLle funds). 

CONSUMER RISK- Ihe probability of accepting an itsm which 
is, in fact, unsatisfactory. 

CONTINGENCY AN.«LYSI3i Repetition of an analysis with different 
qualitative assumptions such as theatsr, or type of con- 
flict, to determine their effects on the results of the 
initial analysis. 

CONTRACT DEFINITION PHASE (cop): The specification, in com- 
peting contracting studies, of detailed technical per- 
formance characteristics, costs, and time-and-coat sched- 
ules for engineering development and production of a 
military and item. 
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CORRELATION: In a ganeral sense in statistics, correlation 

denotes the co-relation ov covariation between two variables. 

COS RELATION COEFFICIENT » A nuabor that attempts to measure 
the interdependency of variables. 

COST. Goods or services used or consumed. 

COST ANALYSIS 2 The systematic examination of cost (total 
resource implications) of interrelated activities and 
equipment to determine the relative costs of alternative 
systems, organisations, and force structures. Cost analysis 
is not designed to provide the precise measurements re- 
quired for budgetary purposes. 

COST CATEGORIES t Three major program cost categories are: 

(1) Research and Development . Those program costs pri- 
marily associated with research and development efforts, 
including the development of a new or improved capability 
to the point of operation. These eoets include equipment 
costs funded under the RDTA2 appropriations and ralatad 
Military Construction appropriation costs. They exclude 
costs that appear is the Military Personnel, Operation 
and Maintenance, and Procurement appropriations. 

(2) Investment. Those program 'costa required beyond 
the development phase to introduce into operational use 
* new capability, to procure initial, additional, or 
replacement equipment for operational forces or to pro- 
vide for major modifications of an existing capability. 

They Include Procurement appropriation costa and all 
Military Construction appropriation costs excert those 
associated with RAO. They exclude ROTAE, Military Par- 
sonnel, and OAM appropriation coats. 

(i) ftpsratlnq. Those program costa necessary to jpirate 
and rtftlntalr. the capability. These coats include Military 
Personnal and 0AM appropriation costa, including funds 
for obtaining replenishment spares fro* stock funds. 

They exclude RDTAE and Military Construction appropriation 
cotta. 

COST S#FtCTlYSx28t ANALYSIS: The quantitative examination 
of alternative prospective systems for the purpose of 
identifying e preferred system and its sssociatad equip- 
ment , organisations, ate. rhe examination aims at 
fuding answers to r question and not at justifying a 
conclusion. The analytical process includes ..rade-offa 
empng alternatives, design of additional alternatives, 
and the measurement of the effectiveness and cost of the 
si tentative*. 
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COST ESTIMATE: Th* estimated cost of a component or aggregation { 

of components. The analysis and determination of coat of 
interrelated activities and equipment is- cost analysis. 

COST ESTIMATING RELATION (CER) : A numerical expression of the 

link between a physical characteristic, resource or activity 
and a particular cost associated with it; e.g., cost of 
aircraft maintenance per flying hour. 

COST INFORMATION REPORTING (CIS) : A uniform system for col- 
lecting and processing cost and related data on major items 
of military aquipmant. Its purpose ia to assist both industry 
and government in planning and managing weapon systems 
development and production activities. 

COST MODEL: An ordered arrangement of data and equations that 
permits translation of physical resources into costs. 

COST SENSITIVITY: The degree to which costs (e.g., total systems 
costs) change in response to varying assumptions regarding 
future weapon system characteristics, operational concepts, 
logistic concepts, and force mix. 

CRITERION: Test of preferredness needed to tell how to choose 
one alternative in preference to another . For each alter 
native, it compares the extent to which the objectives 
are attained with th^ costs or resources used. 

CYBERNETICS: The field of technology involved in the comparative 
study cf the control and intracommunication of information 
handling machines and nervous systems of animals and man 
in order to understand and improve communication. 

DEC*: A collection of punched cards, commonly a complete set 
of cards which have been punched fer a definite service 
or purpose . 

DEPRECIATION: ^eclir i in the value of capital assets over time 
as a result of business operation and/or technological 
innovation. The Internal Revenue Service defines depre- 
ciation as the gradual exhaustion of property employed 
in the trade or business of a taxpayer — such exhaustion 
comprising wear rnd tear, decay or decline from natural 
causes, and various forms of obsolescence-. Accelerated 
depreciation is any formula for depreciation permitted 
by the IRS that provides for a more ranid write-off of 
reproducible assets than would be possible by using rates 
reflecting true economic depreciation. Accelerated depre- 
ciation provides economic incentives for investment in 
plant and equipment . 
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DESIGH ADEQUACY : Probability that the system will success- 
fully accompJ.ich its mission, given that the system is 
operating within design specifications. 

DETERMINISTIC MODEL: A model that permits no uncertainty in 
the magnitudes of either inputs or outputs. An example 
from gunnery is: 

W ■ , where 

1000 

W is the lateral distance at range R; R is the rang®, and 
M is the angular measure in mils of the arc subtended by 
W at range R. For any set of given values for R and M 
there is one and only one value for W. Many deterministic 
models use an average as a constant value input. 

DIGITAL COMPUTER: An electronic device that performs mathe- 
matical operations on numbers which are expressed ab digits 
in some sort of numerical system. 

DIMINISHING RETURNS: An increase in some inputs relative to 
ether fixed inputs will cause total output _o increase; 
but after a point the extra output resulting from the same 
additions of extra inputs ic likely to become less and less. 
This failing off of extra returns is a consequence of the 
fact that the new "doses " of the varying resources have 
less and less of the fixed resources to work with. 

DISBURSEMENTS: The amount of expenditure checks issued and cash 
payments made, net of refunds received. 

DOCUMENTATION* TOt group of techniques necessary for the 

orderly prtaentation, organization and communication of 
recorded specialized knowledge, in order to maintain a 
complete record of reasons for changes in variables. 
Documentation is necessary not so much to give maximum 
utility as to give an unquestionable historical reference 
record . 

DOWN TIME: Total time during which the system is not in accept- 
able operating condition. Down time can in turn be sub* 
divided into a number of categories such as active repair 
time, logistic time, and administrative time. 

DYNAMIC PROGRAMMING; In a multistage decision proce^u, a sys- 
tematic method for searching out that sequence of decisions 
(policy! which maximizes or minimizes some predefined 
objective function. The method is based on Bellman's 
Principle of Optimality which states that: "An optimal 
policy has the property that whatever the initial state 
and initial decision are, the remaining decisions must 
constitute an optimal policy with regard to the state 
resulting from the first decision." 
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ECONOMETRICS t The branch of economics that 'uses mathematics 
and statistics to build and analyse aconoaie models, to 
explain economic phenomena, and to estimate values for 
economic variables. The statistical 'methods used are 
especially designed to deal with time*series data. 

ECONOMIC GROWTH : The sustained increase in the total and per 

capita output of a country as measured by its gross national 
product (in constant prices) or ' other output statistics. 

ECONOMIC LOT SI2E: The coi.t-minimizing'8ize'of ’ order to buy 
or batch to make. 

ECONOMIES OF SCALE: Efficiencies, usually expressed as reduction 
in cost per unit of output, that result 'from increasing 
the size of the productive unit. 

ECONOMY: Using the least amount of resources to attain a given 
output or fixed objective. 

EFFECTIVENESS; The degree or amount of capability to accomplish 
some objective (•) . Various criteria {s.g., targets de- 
stroyed. tonnage moved, etc.) mi got be used to provide 
a measure of this amount of capability. 

EFFICIENCY: Attaining the greatest possible output from a given 
amount of resources. 

EMPIRICAL PROBABILITY t The observed relative frequency; e.g., 
if d is a random sample of size n drawn from a stable 
universe possessing a given trait, the empirical probability 
that an element drawn randomly from that universe is 
estimated to be d/n. 

ESSENTIAL ELEMENT OF ANALYSIS: A question ?pecifically designed 
to obtain data that will provide an anewer in a particular 
problem area, or information required to conduct an eval- 
uation in a particular functional area. 

EXPECTED VALUE: The probability of an event occuring multiplied 
by the payoff associated wish its occurrence. 

EXTERNAL ECONOMIES: Those benefits accruing from a grouping of 
industrial activities or from public facilities. One 
textile plant benefits from the existence of several textile 
plants in a vicinity. 

EXTRAPOLATE: Estimate by trend projection the unknci.'n values 
that lie beyond the range of known values in a series. 
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/AILURK RATE} Tha number of items replaced per unit time due 
to failure e£ that item. 

FEASIBILITY STUDY t (1) A study of the applicability or desir- 
ability of any j#anagemant or procedural system from the 
standpoint of advantages versus disadvantages in any given 
ease; (2) also a study to determine the time at which it 
would be practicable or desirable to install such a system 
when determined to be advantageous r (3) a study to deter- 
mine whether a plan is capable of being accomplished suc- 
cessfully. 

FIELD EXPSRIM2MT t A mode of research involving the response of 
personnel in a field situation or environment to a test 
situation. A field experiment la conducted under stat- 
istically controlled conditions to discover the capabilities 
and limitations of some military plan, organization, or 
material. 

FIXED COSTS i Those elements of cost that do not vary with 
volume of production. 

FIXED POINT ARITHMETIC i (1} A method of calculation in which 
operations take place in an invariant 'manner, and in which 
the computer does not consider the location of the decimal 
point. (2) A type of arithmetic in which the operands 
and results of all arithmetic operations meat be properly 
scaled so as to have a magnitude between certain fixed 
values . 

FLOATING POINT ARITHMETIC} A method of calculation which 
automatically accounts for the location of tha decimal 
point. This la accomplished by handling the number as a 
signed mantissa times the radir raised to an integral 
exponent; e.g., the decimal number +80.3 might be written as 

+0.88300000 x 10 2 . 

FLOW CHART r A graphic rapraeantation of the major steps 

of work in a process. Tha illustrative symbols may represent 
documents, machines, or actions taken during the process. 

The area of concentration is on where or who does what 
rather than on how it is to be done. 

FORCE STRUCTURE ANALYSIS} The analysis of proposed forces to 
obtain a picture of resource implications for planning. 

FORCE STRUCTURE COSTING} The determination of the resource 

implications (manpower, material, support, training, ate.) 
in dollar terms of a given force structure or change to it. 
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FORCASTTNG: Attempting to define possible courses of future 

events. Nay include estimating probabilities associated (, 

with each course of events. 

FORTRAN t A programing language designed for problem which 
can be expressed in algebraic notation, allowing for 
exponentiation and up to three subscripts. The FORTRAN 
compiler is a routine for a given machine which accepts a 
program written in FORTRAN source language and produces ! 

a machine language routine object program. FORTRAN II 1 

added considerably to the powex of the original language . j 

by giving it the ability to define and use almost unlimited j 

hierarchies of subroutines, all sharing a common storage 
region if deeired. Later improvements have added the ebility 
to use boolean expressions, and some capabilities for 
inserting symbolic machine language sequences within e 
source program. 

FREE TINE: Time during which operational use of tho system is 
not required. This my or my not be down time, depending 
on whether the system is in operable condition, 

FREQUENCY DISTRIBUTION: An arrangement of statistical data that 
divides a series of items into claaaea and indicates the 
number of items falling into each class. An exaaqplw is 
the income distribution in which the number of person'' 
falling within each income class ie stated. 

FULL EMPLOYMENT s According to the President's Council of 

Economic Adviaera, the full employment level ie reached 
when no more then four percent of the civilian labor force 
is unemployed. 

s 

GAMING: A method of examining policies and strategies under 
the conditions of a particular scenario, allowing factors 
(human and chance) to vary in the scenario. 

GANTT CHART: A cnart of activity plottsd against time usually 

used to schedule or reserve resources for specific activities. 

i 

GROSS NATIONAL PRODUCT (GNP) : Total value at market prices cf 
all goods and sat vices produced by the nation's economy 
during a period of one calendar year. As calculated 
quarterly by the Department of Commerce, ^roes national 
product is the broadest available measure of the rate of ! 

economic activity > 

GROSS PRIVATE DOMESTIC INVESTMENT: One of the major components 

of GNP, grots privata domestic investment includes annuel j 

outlays for producers' durable goods (machinery and equipment!, 

I private new construction of both residential wnd non-resid- I 

| ential buildings (including those acquired by owner/occupants) , ] 

and ths net change of business investment in Inventories. j 

i i 
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HEURISTIC: Pertaining to systematic trial and error methods 
of obtaining solutions to problems. 

HISTOGRAM: A graphical representation of a frequency distribution 
by means of rectangles whose widths represent the class 
interval:: and whore heights represent the corresponding 
frequencies. 

HOLLERITH: A widely used system of enccding alphanumeric 
inxonaation onto cards, hence "Hollerith" cards is 
synonymous with punch cards. 

■> 

HOMOSTASXS: The dynamic condition of a system wherein the input 
and output are balanced precisely, thus presenting an 
appearance of no change, hence a steady state. 

HUMAN FACTORS ANALYSIS : Individual, behavioral, cultural, or 
social systems and their relation to organisations, pro- 
cedures, and material. 

HUMAN FACTORS ENGINEERING: The development and application of 
scientific methods and knowledge about human capabilities 
and limitations to the selection, design, and control 
operations, environment, and material, and to the selection, 
and training of personnel. 

HYSTERESIS: The lagging in the response of a unit of a system 

behind an increase or a decrease in the strength of a signal. 
It is a phenomenon demonstrated by materials which make 
their behavior a function of the history of the environment 
to which they have been subjected. 

IMPLIED AND INDUCED OUTPUT: Implied output is that which can 
be estimated directly from the nature of the project in- 
cluding all activities without which the projoct could not 
function. Induced output covers the interindustry, or 
intermediate, requirements of those activities that supply 
the project and those which purchase or use its output; 
usually measured by using an input-output table. 

INCOMMENSURABILITY: The inability of two qualities or values 
to be measured by a meaningful relevant common index. 

INCREMENTAL COST: The added coats of a change in the 1-v^l or 
nature of activity. They can refer to any kind of change: 
adding a new product, changing distribution channels, adding 
new machinery. Although they are sometimes interpreted 
to be the same as marginal cost, the latter has a such 
more limited meaning, referring to cost of an added unit 
of output. 
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INDIVIDUAL SYSTEM COSTING: The determination of the total 
resource implications of a system (organization) with- 
out consideration of the interaction of the system 
(organization) as part of a force structure. 

INDEX NUMBER: A magnitude expressed as a percentage of the 
corresponding magnitude in some "base" period. The 
base is usually designated as equal to 100. 

INDIFFERENCE MAP: A two- dimensional graph denoting an indi- 
vidual's preference system with respect to two economic 
quantities. The body of the graph consists of a family 
of ncnintersecting lines convex to the origin. Each 
line of che fami ly represents an equally desirable mix- 
ture of the quantities in question 

INDUSTRIAL DYNAMICS: A philosophy relating to simulation at a systrna 
conceived as a network of flows and feedback loops intaztxnnscting 
a timber of inventories or lewis and rr^. ending to changes in its 
environment. 

INFLATION: A rise in the general level of prices. 

(Pure inflation is defined as a rise in the general 
level of prices unaccompanied by a rise in output.) 

INFORMATION SYSTEM: A combination of personnel, efforts, 
forms, formats, instructions , procedures, data, 
communication facilities and equipment that provides 
an organized and interconnected means — automated, 
manual, or a combination of these— for recording, 
collecting, processing, transmitting and displaying 
information in support of specific functions. 

INFRASTRUCTURE (SOCIAL OVERHEAD CAPITAL': The foundation 
underlying a nation’s, region’s, or community’s 
economy (transportation and communications systems, 
power facilities, schools, hospitals, etc.). 

INPUT-OUTPUT ANALYSIS: A quantitative study of the inter- 
dependence of a group of activitiesbased on the rela- 
tionship between inputs and outputs of the activities. 
The basic tool of analysis is a square input-output 
table, interaction model, for a given period that 
shows simultaneously for each activity the value of 
inputs and outputs, as well as the value ot trans- 
actions within each activity itself. It has been 
applied to the economy and the "indus trios" into 
which the economy can be divided. 

INSTRUCTION : A set of character*, which defines an operation 

together with one or more addresses, or no address, 
and which, as a unit, causes the computer to perform 
the operation on the indicated quantities. 
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INTERACTION t T he difference between e whole end* the simple 
sue of its parts. 

INTERCEPT? Intersection of a line and an axis. 

INTERPOLATE! Estimate the intermediate value in a series 
of m “bars by using a formula that relates the unknown 
value to the pattern of known values in the series. 

INTRINSIC PROBABILITY t The probability that the system is 
opwrating satisfactorily at any point in time when used 
under the stated conditions, where the time ■considered 
is operating time and active repair time. 

XRVB0TMMT COST* The cost beyond the Research and Develop- 
ment phase to introduce a new capability into operational 

use. 

XSOCONTOURSt Graphical raprasantation showing all cootoinatione 
of input# that produce equal output*. 

ITERATIVE j Describing a procedure or process which repeatedly 
executes e series of operations until soe* condition is 
satisfied. An iterative procedure can be implemented 
by a loop in a routine Each iteration or cycle used 
data from the preceding cycle and supplies data to the 
following 

ISOMORPHIC! Similar in pattern. 

JOUST COSTS i Costs that are shared by several departments 
or activities, such ee an airbaaa serving fighter squa- 
drons and transport pxanes; or a dam providing power, 
irri v *ntion, flood control, end recreation. 

JOINT PROBABILITY! The probability that both event A and 
avert B will occur, if a and B ere independent, it is 
the product of their separate probabilities. 

KNOWN UNIVERSE i An idealised Abstraction f **om the real world, 
in which the probabilities of every element in the pop- 
ulation are known. 

LANGUAGE! A system for representing and communicating infor- 
mation which la intelligible to a specific machine. 

Such a language say include instructions which define 
and direct machine operations, and information to be 
recorded by or acted upon by then# machine operations. 

LATIN SQUARES i Experimental designs to avoid compounding 
the effects of inputs while reducing the number of 
observations (and the coet] required to achieve e 
satisfactory confidence level. 
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LEARNING CURVE t The ooet-quan*icy relationships foe sstimating 
costa of equipment. Generally used to predict or da- 
■ cribs the decrease in tha cost of a unit as tho number 
of ’.-’'its produced increasss. 

LEAST-SQUARE* METBQDt A method of fitting a ea lev la tad tread 
to statistical data, so called bacao a# the aw of the 
squared deviations of the calculated free the o bs erve d 
variables is a minimus. "Least squares'* also refare to 
the eriteriou that, when followed, yields this result. 

LIABILITIES t The amounts owed for goods and services received, 
other eseets acquired, and perforaaooe accepted. This 
includes amounts administratively approved ’for payments 
of greats, pons ions , swards, and other indebtedness net 
involving the furnishing of goods and services. 

LINEAR PROGRAMMING t A mathematical method used to determine 
the most affective nl location of limited resources be- 
tween competing demands, Mathematical requirements fer 
applicability of linear programming are? (1) both re- 
sources and activities that use them are non-negative 
quantities; and (2) both the objective (e.g., profit or 
coat) and the restrictions on its attainment are expres- 
sible as a system of linear equalities or inequalities 
(y - «+bx) . Linear prograassing has been employed in 
areas such as the determination of the best product 
mix and che selection of least-coat transportation routes. 

LOGARITHM: The logarithm of a number is the exponent or 
power to which the logarithmic base mast be raised to 
equal that number. 

LOGARITHMIC SCALE. When the vertical axis of s chart is 

laid off in terms of the logarithms of natural numbers 
the arrangement is known as a semilog chart and the ver- 
tical seals is called a log scale. A curve plotted on 
such a chart represents not the numbers in the series 
but the logarithms of these numbers. Changes in the 
slope of such a curve show changes in the percentage 
increase or decrease of the original series. As long 
as there is no change in direction, equal distances on 
the vertical acale correspond to the same percentage 
change in the original serias. 

LOGISTIC TIME: That portion of down tima during which repair 
is delayed solely because of the necessity of mailing 
for a replacement part or other subdivision of the system. 

LOOP t A self-contained series of instructions in which the 
last instruction can aodify and repeat itself until a 
terminal condition is raached. 
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MACH IKE LANGUAGE t A system for expressing information which 
is intelligible to a specific machine. Such a language 
aay include instructions which define and direct machine 
operations, and information to be recorded by or acted 
upon by these machine operations. 

MAINTAINABILITY* Probability that, when maintenance 

action is initiated under statad conditions, a failed 
system will be restored to operable condition within 
a specified total down time. 

MARGINAL COST; REVENUE: Costs incurred or expected to 
be incurred in the production of an additional unit 
of output. Marginal revenue is revenue received 
or expected to be received from the sale of an ad- 
ditional unit of output. Tc maximise its profits, 
a firm has to extend production to the point where 
marginal revenue equals marginal cort. 

MARGINAL OUTPUT OX PRODUCT: The output to be derived 
from the use of an additional unit of a productive 
resource (land, labor, capital, or materials). 

MARGINAL UTILITY t Satisfaction derived from the last 
or additional expenditure. Additional incresmints 
of expenditure for a given product tend to result 
in declining additions cf utility. If utility is 
to be maximised, the as Lisf action derived from the 
last dollar spent on each product or sarvice should 
be the same. 

MASTER PLANNING BUDGET: The estimated cash receipts and 

disburaemants classified as to causes (contra accounts) 
and spread over the future periods in which they are 
predicted to occur. For comparability with other 
plans, each estimated cash flow is converted into 
an expected value, adjusted for risk and diminishing 
utility, and discounted to its present value. 

MATHEMATICAL MODEL t The general characterisation of a 

process, object, or concept, in terms of mathematical 
symbols, which enables the relatively simple manip- 
ulation of variables to be acco«* lished in order to 
determine how the process, object, or concept would 
behave in different situations. 

MATRIX: A rectangular array of terms called elements. 

'It is used to facilitate the study of problems in 
which the relation between these element* is funda- 
■ental. A matrix .* « usually caoable of being subject 
to a sukthematical operation by means of an operator 
or another matrix according to prescribed rule*. 
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MEAN: The most common measure of central tendency equal to 
the turn of the observed quantities divided by the nuabcr 
of observed quantities divided by the n unbar of observations. 

MEDIAN: Halfway point between the two end pornts of an array. 

MISSION: The specific task or responsibility that a person 
or a body of persons is assigned to do or fulfill. 

MISSION RELIABILITY : Probability that under stated conditions, 
the system will operate in the mode for which I** was d 
designed for the duration of a mission, given that it 
was operating in this at the beginning of the mission. 

MODE: A computer aye tern of data representation. The value 

in a set of values that occurs with the greatest frequency. 

MODEL: A simplified representation of an operation, containing 
only those aspects of primary importance to the problem 
under study. The means of raprasentaticn may vary from 
a set of mathematical equations or a computer program 
to a purely verbal description of the situation. In 
cost/effectiveness analysis (or any analysis of choice} , 
the role of the model is to predict the costs that each 
alternative would incur and the extent wo which each would 
attain the objective. 

MONTE CARLO METHOD: Any procedure that involves statistical 

sampling techniques from a distribution of possible out- 
comes for obtaining a. probabilistic approximation to the 
solution of a mathematical or physical problem. Mont? 

Carlo Methods are often used when a great number of 
variables are present, with inter-relationships so extrermly 
complex as to forestall straightforward analytical 
handling. This method generally involve* the use of 
simulated data acquired by putting random numbers thiough 
transformations such that the data imitates significant 
aspects of a situation. 

MQNOTOH IC1TY : In the matnematical sense, sonotonicity refers 
to the constancy of a type of change. For example, if 
a curve is rising (falling) throughout the range of 
interest we say it is a monotonically increasing (decreasing) 

curve 

MOVING AVERAGE: A series cf averages frequently used to 
reduce irregular' ties in « time series by selecting 
a set number of successive items in the series, 
computing the average, then dropping the first item 
and adding the next succeeding one, etc. The ^recess 
is intended to average out random and, 

thcrecy, reveal underlying trends. 
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MUTUALLY EXCLUSIVE: Describing any event the occurrence of 
which precludes the occurrence of all other events 
under consideration. 

NATIONAL INCOME i The Money Measure of the overall annual 
flow of goods and services in a cooaunity equal to the 
sun of condensation of enployees, profits of corporate 
and unincorporated enterprises, net interest, and 
rental income of persons. Is also equal essentially 
to QfP minus (1) allowance for depreciation and other 
capital consumption, and (2) indirect business tax 
and acm-tcx liability to go/enunsut. 

NUMERICAL ANALYSIS i The study of methods of obtaining 

useful quantitative solution* to mathematical problems, 
regardless of whether an analytic solution exists 
c: not, and ths study of the errors and bounds or. 
errors in obtaining such solutions. 

OBJECTIVE; me purpose to be achieved or the position to 
be obtained. Objectives vary with the level of 
suboptimisation of -he study. 

OBJECTIVE FUNCTION: A Mathematical statement of goals, 
usually profit maxiuJ. cation. 

OPERAND; A quanity entering or arising in an instruction. 

An operand may be an argument, a result, a parameter, 
or an indication of the location of the next 
instruction, as opposed to the operation code or symbol 
itself. 

OPE PAT IMG COST: The recurring cost required to operate 
and maintain an operational capability. 

OPERATING TIME : Time during which the system is operating 
in a munner acceptable *o the operator, although 
unsatisfactory operation is sometimes the result of 
judgment of the maintenance man. 

OPERATIONAL READINESS; The probability that, at any point 
in time, the sytem is either operating satisfactorily 
or ready to be placed in operation on deswand when used 
under stated conditions, including stated allowable 
warning time. 

OPERATIONS RESEARCH: The use of analytic methods adopted 
from mathematics for solving operational problems. 

The objective is to provide management with a more logical 
basis for making sound predictions and decisions. 

Among the common scientific techniques used in 
operations research are mathematical programming, 
Statistical the.uy, infer, .ati on theory , game theory, 
monte carlo methods, anc <y*uclnc theory. 
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OPERATOR i A mathematical symbol which represents a 

mathematical process to be performed on an associated 
operand . 

OPPORTUNITY COST : The cost of foregone opportunities; the 
sacrificed amount of money, equipment, or unite of 
production that could have been realized by a separate 
course of action (alternative) with the same time and 
effort expended. 

OPTIMIZATION: The attainment of the beat possible result, 
i.e., the maximization (minimization) of some desirable 
(undesirable) criterion measure, subject to the 
constraints imposed on the choice of solutions, 

ORDINATE: The vertical distance on a graph; i.e., the 
distance from the horizontal axis. 


PARAMETER: A constant or a variable in mathematics which 

remains constant during some calculation. It ic generally 
a definable characteristic of an item, device, or system. 

PARAMETRIC ANALYSIS: Parametric analysis assumes a ranje 
of values for each parameter which will bracket the 
expected values of that parameter, and a solution to 
the problem is obtained for ea^h set of assumed par.oneter 
values. 

PAYOFF; The -gain to be derived if a particular course of 
events develops. 

°ERIPHERAL EQDIP?" i ’NT: The auxiliary machines which may be 

placed und^i t,V> control of the central computer. Examples 
of this are card readers, card punches, magnetic tape 
feeds, and high-speed printers, 

PERMUTATIONS: The number of possible sequences of n items 
taken c at a time. 

a ni ? <c } ''' 7iFcJT““ c (£>c! 
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PLANING: The ae lectio:, of courses of action through a 
systematic consideration of alternatives in order to 
attain organizational objectives. 

PLOTTER: A visual display or board controlled by a 

computer in which a dependent variable is graphed by 
an automatically controlled pen or pencil as a function 
of one or more variables. 

PRESENT VALUE: The estimated present worth of a stream of 
future benefits or costs arrived at by discounting the 
future values, using an appropriate interest rate. 

PROBABILISTIC MOSEL: A model that makes allowances for 

randomness in one or more of the factors that determine 
the outputs of the model. For example, an inventory 
model that optimizes an inventory policy to avoid inventory 
shortages is probabilistic if intakes explicit account 

of uncertainty over time, in the distribution of demands 
on the inventory. On the other hand, the model 
would be deterministic if it assumed that the rate of 
demand against the inventory is always the same (usually 
the estimated average demand). In this example, a 
deterministic model would most probably give answers 
that would lead to bad inventory policies. However, 
there are times when the use of a deterministic model 
in a probabiliaitc situation does no harm. 

PROBABILITY: A number between 0 and 1 that, when assigned 
to an event or occurence, expresses the likelihood that 
the event will occur. 

PROBABILITY DISTRIBUTION ; Tables showing relative 

frequencies of each subset into which the total population 
is divided ; a table showing the probability or occurrence 
of each possible value. 

PRODUCER'S RISK: The probability of rejecting an item which 
is, in fact, satisfactory. 

PROGRAM: (1) A plan or scheme of action designed for the 

accomplishment of a definite objective that is specific 
as to the time-phasing of the work tc be done and the 
means proposed for its accomplishment, particularly in 
quantitative terms, with respect to manpower, material , 
and facilities requirements: thus a program provides a 
basis for budgeting; (2) a segment or element of a 
complete plan; (3) a budget account classification. 
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PROGRAMMING: The process of translating planned- Military force 
requirements into specific tiue^phased, scheduled actions, 
and of identifying in relatively praise terns the 
resources required. It is the bridge between planning 
and budgeting. 

QUANTIFY: To qualify with respect to quantity. In analysis, 

to translate observed physical relationships into analogous 
mathematical relationships . 

QUEUING THEORY: A theory that deals with the analysis of costs 
and effectiveness when items appear with some randomness 
for processing at a facility with a capacity for processing 
simultaneously fewer items that may be waiting at a 
given time. The costs are costs of waiting and of providing 
the capacity to reduce the amount of waiting. Examples 
of queuing problems are* (1) determination of a number 
of checkout counters at a supermarket that minimises 
the sum of costs of customer dissatisfaction if they must 
wait in line and costs of providing additional checkers/ 

(2) determination of the capacity of communications 
capacity and of delays in the processing of messages. 

RANDOM ACCESS: Pertaining to the process of obtaining 

information from or placing information into computer 
storage where the time required for such access is 
independent of the location of the information most 
recently obtained or placed in storage. 

RANDOM NUMBERS: A sequence of digits in which each digit has 
an equal probability of occurring in eaeh position, 
wholly independent of which digits appear elsewhere in 
the sequence. 

RANDOM number GENERATOR: A special computer routine designed 
to produce a random number or series of random numbers 
according to specified limitations. 

RANDOM SAMPLE: A sample selected, from a population to be 

tested, in such a manner that every element in the popu~ 
lation has an equal chance of being chosen for the sample. 

RANDOM VARIABLE: A function defined on a sample space. 

It is called discrete if it assume* only a finite 
or denumerable number of vaiut« and continuous if it 
assumes a continuum of values. 

R CHARTS: Charts of the range of small samples, useful, in 
moritoring change in dispersion in the product 
of a system. 
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REAL TIME OPERATION: The use of the computer as a element 

of a processing system in which the times of occurrence 
of data transmission are controlled by other pert ions 
of the system, or by physical events outside the system, 
and cannot be modified for convenience in computer 
programming. 

REDUNDANCY: The existence of more than one means for 

accomplishing a given task, where all means must fail 
before there is an overall failure of the system. 

RELATIVE FREQUENCY: The ratio of the number of observations 
(elements) in a class (subset) to the total number 
of observations constituting a population (universe or 
set) . 

RELIABILITY : The probability that the system will perform 
satisfactorily for at least a given period of time 
when used under stated conditions. 

REORDER LE/EL: The inventory balance at which a replacement 
order is placed. 

REPAIRABILITY : The probability that a failed system will be 
restored to operable condition within a specified 
active repair time. 

REPROGRAMMING: The reappli cation of funds between budget 

activities or line items within a single appropriation 
account . 

REQUIREMENT: The need or demand for personnel, equipment, 
facilities, other resources or services, expressed in 
specific quantities for specific time periods. 

RESEARCH AND DEVELOPMENT (R&D) : Basic and applied research 
in the sciences and engineering, and the design and 
development of prototypes and processes. Excludes 
routine product testing, market ■'esearch, sales promotion, 
sales service, and other non-teci oological activites or 
technical services. 

Basic research includes original investigations for 
tKe”advanceroent of scientific knowledge that do not have 
specific practical objectives. 

Applied research is the practical application of knowledge, 
material and/or techniques toward a solution to an 
existent or anticipated military or technological 
requirement. 
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Developmen t includes tec/*niccl act 4 vities of a nonroutin* 
nature concerned with translating research findings or 
other scientific knowledge into prefects or processes* 
Development does not include routine technical services 
or other activities excluded from the above definition 
of research and development. 

RESEARCH AMD DEVELOPMENT • kSD ) COSTS: The cost of 

developing a new capability to the point where it is 
ready for procurement for operational units. 

RESOURCE IMPACT: The cost of adopting a course of action stated 
in measurable terms. Resource impacts cannot always be 
reduced to dollar terms. 

REVOLVING FUND: A fund established to finance a cycle of 

operations to which reimbursements and collections are 
returned for reuse in a manner' such as to maintain the 
principle of the fund; eg., working-capital fund, 
stock fund. 

RISK: As used in cost-effectiveness analysis and operations 
research, a situation is characterized as risk if it is 
possible to describe all possible outcomes and to 
assign meaningful objective numerical probability weights 
to each one. For example, an action might lead to this 
risky outcome: a reward of $10 if a "fair” coin comes 
up heads, and a loss of $5 if it comes up tails. 

Another example, 50% of all missiles f. : red can be expected 
to land within one CEP of the target. 

ROUTINE: A set of coded instructions arranged in proper sequence 
to direct the computer to perform a desired operation 
or sequence of operations. 

SAMPLE SPACE: The range of feasible solutions. 

SAMPLXNG: The process of determining characteristics of a 
population by collecting and analyzing data from a 
representative segment of the population. 

SAMPLING ERROR: That part of the variation in the data 

resulting from an experiment that is not explained by 
the variation in the factors controlled during 
the experimentation. 

SCENARIO: A word picture of a fixed sequence of events in a 
defined environment. 


A- 24 


AMCP 706-191 


SENSITIVITY ANALYSIS: Repetition of an analysis with different 
quantitative values for cost or operational assumptions 
or estimates such as hit-kill probabilities, activity 
rates, or R*D costs, in order to determine their effects 
for the purposes of comparison with the result* of the 
basic analysis. I Z a small change in an assumption 
results in a proportionately or greater change in the 
the results, then the results are said to be sensitive 
to that assumption or parameter. 

SETS: A collection of items (elements) chosen as pertinent. 

SHADOW PRICE: The shadow price of a factor is a measure of its 
opportunity cost or its marginal product. For example, 
when unemployment is widespread, the opportunity cost 
of labor may be near zero, so that the shadow price of 
labor may be well below the prevailing wages of 
those workers who are actually employed. 

SIMULATION: The representation of physical systems and 
phenomena by computers, models, or other equipment. 

The model or computer representation is manipulated to 
imitate significant aspect!, of a situation. 

SPURIOUS CORRELATION: Accidental correlation having no 

causative basis and without expectation of continuance. 

STANDARD DEVIATION: A measure of the dispersion of observed 
data. Mathematically, it is the positive square root 
of the variance. 

STANDARD ERROR: The standard deviation of a group of measures 
of the same characteristics (often termed a "statistic" 
or a "parameter"), each obtained from a distinct sample 
drawn from a larger "universe" or "population". 

STATISTICAL BIAS: If some samples or observation dar. are 

more likely to be chosen than others, or if subjective 
methods are used in selecting sample data, the results 
are considered biased. 

STATISTICAL DECISION T’tSORY : Theory dealing with logical 

analysis of choice among courses of action when (1) the 
consequence of any course of action will depend upon the 
"state of the world", (2) the true state is as yet unknown, 
but (3) xt is possible at a cost to obtain additional 
information about the rtate. 
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STOCHASTIC PROCESS* The statistical concept undarlying tha 
prediction of tha condition of an element of a 
larger group whan tha probable average condition 
of t$e larger group is known. For example, assume that 
an armored division, under certain circumstances, has 
on the average a certain number of tanks dead lined for 
unscheduled maintenance. The probability that 
any given tank under the same circumstances will be dead* 
lined for unscheduled maintenance on a specific day is 
described by a stochastic process. 

STOCXOUT COST* The cost due to disrupted schedules or to 
inability to satisfy customers because items 
ordinarily stodked are noc available. 

STORAGE TIME* The time during which the system is 

presumed to be inoperable condition, but is being held 
for emergency; e.g,, as a spare. 

SUBOPTIMIZATION* Optimization refers to a selection of a set 

of actions that maximize the achievement of some objective 
subject to all of the real constraints that exist. 

For example, one may optimize a choice of weapons for 
achieving certain objectives of a decision but within 
the given constraint of a certain maximum cost of a 
division. But one suboptimizes on achievement cf the 
division objective if he is given discretion only over 
the amount and kind of armor and ia given a maximum 
amount of money to spend on armor. The objective he 
maximizes directly may be only the mission of ar>ior in 
the division's objective. Such a suboptimization will 
yield something inferior to an optimized expenditure 
on different kinds of armor if the total budget for 
armor givan to the auboptimizer is really not optimal, 
or if there are interdependencies between decisions on 
armor and decisions on other things that are outside the 
discretion of the perron suboptimizing on armor. 

SUBROUTINE: The set of instructions necessary to direct 

the computer to carry out a well defined mathematical 
or logical operation. 

SUBSET: A collection wholly contained within a larger collection; 
a group of elements constituting part of a universe. 

SYMBOLIC LOGIC: The study of formal logic and mathematics 
by means of a special written language which seeks to 
avoid the ambiguity and inadequacy of ordinary language. 

SYSTEM: Weapon system \s composed of equipment, skills and 
techniques , the composite of which forms an instrument 
of combat. The complete weapon system includes a'l 
related facilities, equipment, materials, services, and 
peraonnel required solely for its operation, so that the 
instrument of combat becomes a self-sufficient unit 
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of striking power in its intended operational environment. 
Support system is a composite of equipment, skills, and 
techniques that, while not an instrument of combat, is 
capable of performing a clearly defined function in support 
of a mission. A complete support system includes all 
related facilities, equipment, materials, services, and 
personnel required for operation of the system, so that 
it can be considered a self-sufficient unit in its intended 
operational environment. 

SYSTEMS ANALYSIS (SA) t A formal inquiry intended to advise 

a decision maker on the policy choices involved in a major 
decisions. In DoD a systems analysis may be concerned 
with such matters as weapon development, force posture 
design, or the determination of strategic objectives. 

To qualify as a system analysis a study must look at 
an entire problem as a whole. bnaractvtristically, it 
will involve a systematic investigation of the decision- 
maker's objectives and of the relevant criteria; 
a comparison— quantitative when possible— of the costs, 
effectiveness, and risks associated with the alternative 
policies or strategies f >r achieving each objective; and 
an attempt to formulate additional alternatives if those 
examined are deficient. 

SYSTEMS EFFECTIVENESS; The probability chat the system can 
successfully meet an operational demand within a given 
time when operated under specified conditions. 

SYSTOS APPROACH: The art cf eMffliring the entire context within which 
the item of interest will function. 

TCHEBYCHEFF ' S THEOREM; The proportion of the observations 
falling between -kc and +ko is at least as large as 
l-(l/k2) regardless of the distribution. 

TIME -PHASED COSTS: A presentation of the cost results 
broken down by the time period in which the costs 
occur rather than a single total cost figure. 

TOTAL OBLIGATION AUTHORITY (TOA) : The cost allocated to a 

given system or organization. This cost when related 

to a specific time period, for example a year, represents 

obligations that can be incurred during that year and 

not necessarily expenditures. The total obligation authority 

for a specific year to furrish a house is “he cost of 

what can be ordered during that year ever, if deliveries 

and payments are made in later years. 

TOTAL SYSTEM COST: The total R&D, I: _otment, and Operating 
Cost" (for a specified number of years of operation) 
required to develop, procure, and operate the particular 
weapon system. 
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TRANSFORMS Th« derivation of a new body of data from a given 
one according to specific procedures, often leaving some 
feature invariant. 

TROOP TESTs A troop test is a test conducted in the field, 
using TOE units or units organized under proposed TOE, 
to evaluate current or proposed doctrine and organisations. 
Material is considered in the conduct of troop testa 
only insofar as material affects the doctrine or 
organization being evaluated. 

TYPE 1 ERROR: The belief that something true is false. 

TYPE II ERROR: The belief that something false is true. 

UNCERTAINTY: A situation is uncertain if there 1 * no objective 
basis for assigning numerical probability weights to the 
different possible outcomes or thera is no way to describe 
the possible outcomes. For example, the probability 
of a foreign nation continuing to furnish the U.S. with 
base rights is an uncertainty 

UTILITY: a personal subjective value of a tangible cr 
intangible commodity. 

VALUE ADDED BY MANUFACTURE: That part of value given products 
shipped actually created within a given industry. The 
unadjusted series is calculated by subtracting the cost 
of materials, supplies, containers, fuel, purchased electric 
energy, and contract work from the value of shipments. 

The adjusted series, which is more inclusive, is equal 
to the unadjusted series plus: (1) value added by 
merchandising operations, and (2) the net change in 
invent''’'- os (both finished goods and work-in- progress) 
between the beginning and end of the year. (Value added 
is almost: free statistically from the duplication of values 
existing in the value of shipments and approximates the 
net value of manufacturers). 

VARIABLE CC3TS: Those costs that vary with the volume of output 
as contrasted with fixed costs, which do not vary with 
output . 

VARIABLES: General numbers, auch as x or y which may take on 

many values or which may hove conditional fixed values 
as in x* + 2x = 19. 

VARIANCE: A measure of d-.spersion of a frequency distribution 

computed by summing the squares of the difference 
between each observation and the arithmetic mean of the 
distribution and then dividing by the number of 
observations . 
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IOUI GAME i A simulation, by whatever swans, of a military 
operation involving two or more opposing forces, 
conducted using rules, data, and procedures 
designated to depict an actual or assumed real life 
situation. 
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TABI£ C-4 


9 Distribution: Upper 1 Percent Points 


Degress of freedom la Burners. tor v 


15 i 8.6831 


12.246 i 5.5466 
8.6491 
8.0215 
7-5594 
7-2057 
6.9266 
6.7010 
6.5149 
5.3589 
6.2262 
6.1121 
6.0129 
5-9259 
5-8489 
5-7804 
5-719C 
5-6637 
5.6136 
5.5680 

5.5263 
5-4381 
5.4529 
5.4205 
5.3904 


3 


5403.3 

99.166 

29.457 

16.694 

12.060 

9.7795 

8.4513 

7.5910 

6.9919 

6.5523 

6.2167 

5.9526 

5.7394 

5.5639 

5.4170 

5.2922 

5.1850 



4 


5 

5624.6 

99.249 

28.710 

576 

99. 

28. 

3.7 

299 

237 

f 

11.392 

■O- 

10 e 

967 

9.1483 

8.7 

459 

7.8467 

7.4604 

7.0060 

6.6 

318 

6.4221 

6.0 

569 

5.9943 

5.6 

363 

5.6683 

5 3 

160 

5-4119 

5.0 

643 

5-2053 

4.8 

616 

5.0354 

4.6 

950 

4.8932 

4.5 

556 

4.7726 

4.4 

374 

4.6690 

4.3 

359 

4-5790 

4.2 

479 

4.5003 

4.1 

708 

4.4307 

li P 

4.1 

027 

4.3134 

3.9 

880 

4.2635 

3.9 


4.2134 

3.8951 

4.1774 

3-8550 


3.8183 

4.1056 

3.7848 

. 4.0740 

3.7539 

4.0449 

3-7254 

4.0179 

3.6990 

3.82B3 

3.6491 

3.3 

389 

3.3192 

3* A 

3.0 

f 30 

173 



4.4558 

4.3183 

4.2016 

4.1015 

4.0146 

3.9386 

3.8714 

3.8117 


J. I 

3.7102 

3.6667 

3.6272 

3.5911 

3.5580 

3.5276 

3.4995 

3.4735 

3-2910 

3-1187 

2-9559 

2.8020 


5928.3 

99-356 

27.672 

14.976 

10.456 

8.2600 

6.9928 

6.1776 

5-6129 

5.2001 

4.8861 

4.6395 
4.4410 
4.2779 
4.1415 
4.0259 
3.9267 
3.8406 
3.7653 
3.6987 

3.6396 
3.5667 
3-5290 
3.4959 
3.4568 
3.4210 
3.3882 
3.3581 
3.3302 
3.3045 

3.1238 

2.9530 

2.7918 

2.6393 


4.3021 

4.1399 

4.0045 

3.8896 

3.7910 

3.7054 

3.6305 

3-5644 

3.5056 

3.4530 

3-4057 

3.3629 

3.3239 

3.2884 

3.2556 

3.2259 

3,1982 

3.1726 

2.993C 

2.8233 

2.6629 

2.5113 


9 


6022.5 

99.388 

27-345 

14.659 

10.158 

7.9761 

6.7188 

5.9106 

5.3511 

4.9424 

4.6315 

4.3875 

4.1911 

4.0297 

3.8948 

3.7804 

3.6922 

3.5971 

3.5225 

3.4567 

3-3961 

3.3458 

3.2986 

3.2560 

3,2172 

3.1818 

3.1494 

3.1195 

3.0920 

3.0665 

2.8876 

2.7185 

2.5586 

2.4073 
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TABLE C-4 (continued) 

P Distribution: Upper 1 Fsraent Points 


Degrees of Freedom In Numerator Vj 


u 

10 

12 

15 

20 

24 

30 

40 


6055.8 

6106.3 

6157.3 

6208.7 

6234.6 

6260.7 

6286.8 


99.399 

99.416 

99.432 

99-449 

99.458 

99.466 

99.474 


27.229 

27.052 

26.872 

26.690 

26.598 

26.505 

26.411 


14.546 

14,37* 

14.198 

14.020 

13.929 

13.838 

13.745 

5 

10.051 


9.7222 

9.5527 

9.4665 

9.3793 

9.2912 

6 

7.8741 

7.7183 

7.5590 

7.395* 

7.3127 

7.2285 

7.1432 

7 

J.6201 

6.4691 

6.3143 

6.1554 

6.0743 

5.9921 

5.9084 

8 

5.8143 

5.6668 

5.5151 

5.3591 

5.2793 

5.1981 

5.1156 

9 

5.2565 

5.1114 

4.9621 

4.8080 

4.7290 

4.6486 

4.5667 

10 

4.8492 

4.7059 

4.5582 

4.4054 

4.3269 

4.2469 

4.1653 

11 

4.5393 

4.39"4 

t.2509 

4.0990 

4,0209 

3-9411 

3.8596 

12 

4.2961 

4.1553 

4.0096 

3.8584 

3.7805 

3.7008 

3.6192 

13 

4.1003 

3.9603 

3.8154 

3.6646 

3.5868 

3.5070 

3.4253 

It 

3.9394 

3.8001 

3.6557 

3.5052 

3.4274 

3.3476 

3.2656 

15 

3.8049 

3.6662 

3.5222 

3.3719 

3.2940 

3.2141 

3.1319 

16 

3.6909 

3.5527 

3.4089 

3.2588 

3.1808 

3.1007 

3.0182 

17 

3.5931 

3.4552 

3.3117 

3.1615 

3.0835 

3.0032 

2.9205 

18 

3.5082 

3.3706 

3.2273 

3.0771 

2.9990 

2.9185 

2.8354 

19 

3.4338 

3.2S65 

3.1533 

3.0031 

2,9249 

2.8442 

2.7608 

20 

3.3682 

3.2311 

3-0880 

2.9377 

2.8594 

2.7785 

2.6947 

21 

3.3098 

.3.1729 

3.0299 

2.8796 

2.8011 

2.7200 

2.6359 

22 

3.2576 

3.1209 

2.9780 

2.8274 

2.7488 

2.6675 

2.5831 

23 

3.2106 

3.0740 

2.9311 

2.7805 

2.7017 

2.6202 

2.5335 

£4 

3.1681 

3.0316 

2.3887 

2.7380 

2.6591 

2.5773 

2.4923 

25 

3.1294 

2.9931 

2.8502 

2.6993 

2.6203 

2.5383 

2.4530 

26 

3.0941 

2.5579 

2.8150 

2.6640 

2.5848 

2.5026 

2.4170 

27 

3.0618 

2.9256 

2.7827 

2.6316 

2.5522 

2.4699 

2.3840 

28 

3.0320 

2.8959 

2.7530 

2.6017 

2.5223 

2.4397 

2.3535 

29 

3.0045 

2.8685 

2.7256 


2.4946 

*.4118 

2.3253 

30 

'*.'.9791 

2.8431 

2.7002 

2.5487 

2.4689 

2.3860 

2.2992 

40 

2. 8005 

2.6648 

2.5216 

2.3689 

2. 2880 

2.2034 

2.1142 

60 

2.6318 

2.4961 

2.3523 

2.1978 

2.1154 

2.0285 

1.9360 

12C 

2.4721 

2.3363 

2.1915 

2.0346 

1.9500 

i.eeoo 

1.7628 

m 

2.3205 

2.1848 

2.0385 

1.8783 

1 7908 

1.6964 

1.5923 


60 


180 


6313.0 

95.483 

86.316 

13.658 

9.8080 


6339.4 

99.491 

85.881 

13.558 

9.1118 


6366.0 

99.501 

86.185 

13.463 

9.0204 


7.0568 

5.8386 

5.0316 

4.4831 

4.0819 

3.7761 

3.5355 

3.3413 

3.1313 

3.0471 

2.9330 

2.8348 

2.7493 

2.6742 

2.6077 

2.5484 


6.9690 
5.7572 
4.9460 
4.3978 
3.9965 
3 6904 
3.4494 
3.2548. 
3.0942 
2.9595 
2.8447 
2.7459 
2.6597 
2.5839 
2.5168 
2.4568 


6.8801 

5.6495 

4.8588 

4.3105 

3.9090 

3.6025 

3.3608 

3.1654 

3.0040 

2.8684 

2.7528 

2.6530 

2.5660 

2.4893 

2.4812 

2.3603 


2.4951 

2.4471 

2.4035 

2.3637 

2.3273 

2.2938 

2.2629 

2.2344 

2.2079 


2.4029 

2.3542 

2.3099 

2.2695 

2.2325 

2.1984 

2.1670 

2.1378 

2.1107 


2.3055 

2.2559 

2.2107 

2.1694 

2.1315 

2.0965 

2.0642 

2.0342 

2.0062 


2.0194 

1.8363 

1.6557 

1.4730 


1.9272 

1.7263 

1.53jO 

1.3246 


1.8047 

1 . 


1.000 
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1 

2 

3 

647.79 

799.50 

864.16 

36.506 

39.000 

39.165 

17.443 

16,044 

15.439 

12.218 

10.649 

9-9792 

10.007 

8.4336 

7.7636 

8.8131 

7.2598 

6.5988 

8.0727 

6.5415 

5.8898 

7.5709 

6.0595 

5.4160 

7.2093 

5-7147 

5.0781 

6.9367 

5-4564 

4.8256 

6.7241 

5-2559 

4.6300 

6.5538 

5.0959 

4.4742 

6.4143 

4.9653 

4.3472 

6.2979 

4.856? 

4.2417 

6.1995 

4.7650 

4.1528 

6.1151 

4.6867 

4.0768 

6.0420 

4.6189 

4.0112 

5.9781 

4.5597 

3.9539 

5-9216 

4.5075 

3.9034 

5-8715 

4.4613 

3.8587 

5.8266 

4.4199 

3.8188 

5.7863 

4.3828 

3.7829 

5.7498 

4.3492 

3.7505 

5.7167 

4.318? 

3.72U 

5.6864 

4.2909 

3.6943 

5.6586 

4.2655 

3.6697 

5.6331 

4.2421 

3.6472 

5.6096 

4.2205 

3.6264 

5.5878 

4,2006 

5.6072 

5.5675 

4.1821 

3.5894 

5.4239 

4.0510 

3.4633 

5.2857 

3. 5253 

3-3425 

5.1524 

3.8046 

3.2270 

5.0239 

3.6839 

3.1i6l 


TABLE C-4 (continued) 


7 Distribution: Upper 2. 3 Feroent Points 


Degress of Treedon in Huaerator v 


899.58 
39.248 
15.101 
9.6045 
7.3879 
6.2272 
5.5226 
5.0526 
4.7181 
4.4683 
4.2751 
4.1212 
3.9959 
3.8919 
3.8043 
3.7294 
3.6648 
3.6083 
3.5587 
3.5147 
3.4754 
3.4401 
3 4083 
3.3794 
3.3530 
3.3209 
3.3067 

3.2863 

3.2674 

3.2499 

3.1261 

3.0077 

2.8943 

2.7858 


5 


921.85 

39.298 

14.885 

9.3645 

7.1464 

5.9876 

5.2852 

4.8173 


4.2361 


3.1048 

3.0828 

3.0625 

3.0438 

3.0265 

2.9037 

2.7863 


937.11 

39.331 

14.735 

9.1973 

6.9777 

5.8197 

5.1186 

4.6517 

4.3197 

4.0721 

3.8807 

3.7283 

3.6043 

3.5014 

3.4147 

3.3406 

3.2767 

3-2209 

3.1718 

3.1283 

3.0895 

3.0546 

3.0232 


2.9447 

2.9228 

2.9027 

2.8840 

2.8667 

2.7444 

2.6274 

2.5154 

2.4082 


948.22 

39-355 

14.624 

9.0741 

6.8531 

5.6955 

4.9949 

4.5286 

4.1971 

3.9498 

3.7586 

3.6065 

3.4827 

3.3799 

3.2934 

3.2194 

3.1556 

3.0999 

3.0509 

3.0074 

2.9686 

2.9338 

2.9024 

2.8738 

2.8478 

2.8240 

2.8021 

2.7820 

2.7633 

2.7460 

2.6238 

2.5068 

2.3948 

2.2875 


956.66 

39.373 

14.540 

8.9796 

6.7572 

5.5996 

4.8994 

4.4332 

4.1020 

3.8549 

3.6638 

3.5116 

3.3880 

3.2853 

3.1987 

3.1248 

3.0610 

3.0053 

2.9563 

2.5128 

2.8740 

2.8392 

2.8077 

2.7791 

2.7531 

2.7293 

2.7074 

2.6872 

2.6686 

2.6513 

2.5289 

2.4X17 

2.2994 

2.1918 


9 


963 28 
39.387 
14.473 
8.9047 
6.6810 
5.5234 
4.8232 
4.3572 
4.0260 
3.7790 
3.5879 
3.4358 
3.3120 
3.2093 
3.1227 
3.0488 
2.9849 
2.9291 
2.8800 
2.8365 
2.7977 
2 .7628 
2.7313 
2.7027 
2.6766 
2.6528 
2.6309 
2.6106 
2.5919 
2.5746 

2.4519 

2.3344 















Degrees of Freedom In Denominator 
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TABLE C-4 (continued) 

f Distribution: Upper 2.5 Pereent Points 


Degrees of Freedom in Wumerator 


10 

12 

15 

20 

24 

30 

40 

60 

120 

m 

968.63 

976.71 

984.87 

993.10 

997.25 

1001.4 

1005.6 

1009.8 

1014.0 

1018.3 

39-398 

39.415 

39.431 

39.448 

39.456 

39.465 

39.473 

39.481 

39.490 

39.498 

14.419 

14.337 

14.253 

14.167 

14.124 

14.081 

14.037 

13.992 

13.947 

13.902 

8.8439 

8.7512 

8.6565 

8 5599 

8.5109 

8.4613 

8.4111 

8.3604 

8.3092 

8.2573 

6.6192 

6.5246 

6.4277 

6.3285 

6.2780 

6.2269 

6.1751 

6.1225 

6.0693 

6.0153 

5.4613 

5.3662 

5.2687 

5.1684 

5.1172 

5.0652 

5.0125 

4.9589 

4.9045 

4.8491 

4.7611 

4.6658 

4.5678 

4.466? 

4.4150 

4.3624 

4.3089 

4.2544 

4.1989 

4.1423 

4.2951 

4.1997 

4.1012 

3.9995 

3.9472 

3.8940 

3.8398 

3.7844 

3.7279 

3.6702 

3-9639 

3.8682 

3.7694 

3.6669 

3.6142 

3.5604 

3.50f5 

j.4493 

3.3918 

3.3329 

3.7168 

3.6209 

3.5217 

3-41.86 

3.3654 

3.3110 

3,2554 

3.1984 

3.1399 

3.0798 

3.5257 

3.4296 

3.3299 

3.2261 

3.1725 

3.1176 

3.0613 

3.0035 

2.9441 

2.8828 

3.3736 

3.2773 

3.1772 

3.0728 

3.0187 

2.9633 

2.9063 

2.8478 

2.7874 

2.7249 

3.2497 

3.1532 

3.0527 

2.9477 

2.8932 

2.8373 

2,7797 

2.7204 

£.6590 

2.5955 

3.1469 

3.0501 

2.9493 

2.8437 

2.7888 

2.7324 

2.6742 

2.6142 

2.5519 

2.4872 

3.0602 

2.9633 

2.8621 

2.7559 

2.7006 

2.6437 

2.5850 

2.5242 

2.4611 

2.3953 

2.9862 

2.8890 

2.7875 

2.6808 

2.6252 

2.5678 

2.5085 

2.4471 

2.3831 

2.3163 

2.9222 

2.8249 

2.7230 

2.6158 

2.5598 

2.5021 

2.4422 

2.3801 

2.3153 

2.2474 

2.8664 

2.7689 

2.6667 

2.5590 

2.5027 

2.4445 

2.3842 

2.3214 

2.2558 

2.1869' 

2.8173 

2.7196 

2.6171 

2.5089 

2.4523 

2.3937 

2.3329 

2.2695 

2.2032 

2.1333 

2.7737 

2.6758 

2.5731 

2. 46*5 

2.4076 

2.3486 

2.2873 

2.2234 

2.1562 

2.0853 

2.7348 

2.6368 

2.5339 

2.4247 

2.3675 

2.3082 

2.2465 

2.1819 

2.1141 

2.0422 

2.6998 

2.6017 

2.4984 

2.3890 

2.3315 

2.2716 

2.2097 

2.1446 

2.0760. 

2.0032 

2.6682 

2.5699 

2.4665 

2.3567 

2.2989 

2.2389 

2.1763 

2.1107 

2.0415 

1.9677 

2.6396 

2.5412 

2.4374 

2.3273 

2.2693 

2.2090 

2.1460 

2.0799 

2.0099 

1.9353 

2.6135 

2,5149 

2.4110 

2.3005 

2.2422 

2.1816 

2.1183 

2.0517 

1.9811 

1.9055 

2.5895 

2.4909 

2.386/ 

2.2759 

2.2174 

2.1565 

2.0928 

2.0257 

1.9545 

mm 

2.5676 

2.4688 

2.3644 

2.2533 

2.1946 

2 . 1334 

2.0693 

2.0018 

1.9299 

1.8527 

2.5473 

2.4484 

2.3438 

2.2324 

2.1735 

2.1121 

2.0477 

1.9796 

1.9072 

1.8291 

2.5286 

2.4295 

2.3248 

2.2131 

2.1540 

2.0923 

2.0276 

1.9591 

1.8861 

1.8072 

2.5112 

2.4120 

2.3072 

2.1952 

2.1359 

2.0739 

2.0089 

1.9400 

1.8664 

1.7867) 

2.3882 

2.2832 

2.1819 

2.0677 

2.0069 

1.942? 

1.8752 

1.8028 

1.7242 

1.6371’ 

2.2702 

2.3692 

2.0613 

1.9445 

1.8817 

1.8152 

1.7440 

1.6668 

1.5810 

1 . 4822 

2.1570 

2.0^48 

1.9450 

1.8249 

1.7597 

1.6899 

1.6141 

1.5299 

1.4327 

1 . 3104 

2.0483 

1.9447 

1.8326 

1.7085 

1.6402 

1.5660 

1.4835 

1,3883 

1.2684 

1.0000 



'I 

J 

















TABLE C-4 (continued) 
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7 Distribution! Upper 3 Psrosrr. Points 


Ssgrets of Prstdcn In Jtaasmtor, v 


3 I * I 5 6 


0 13 

| 14 

1 15 
| 16 

5 18 

I 19 

| 83 

» 21 

u 

o I 

• 23 


3*9883 
3.8653 
3.8056 
3.7389 
3-6883 
3-6337 
3.5915 
3.5546 
3.5219 
3.4928 
3.4668 
3,4434 
3.4881 
3.4088 
3.385£ 
3-3690 
3.3^41 
3.3404 
3.3277 
3.33 58 


3.8686 

3.7083 

3.5874 

3.4903 

3.4105 

3.3439 

3.2874 

3-2389 

3.1968 

3.1599 

3.1274 

3-0984 

3.0725 

3.0491 

3.0*80 

3.0068 

2.9912 

2.9751 

2.9604 

8.9467 

8.9340 

2.9223 


3.2317 2.8387 

3.1504 2^7581 

3.0718 2.6802 
3.8415 i 8.9957 2.6049 


224.58 

19.247 

9.1172 

6.3883 

5.1922 

4.5337 

4.1203 

3.8378 

3.6331 

3.4780 

3.3507 

3.2592 

3.1791 

3.1122 

3.0556 

3.0069 

2.9647 

2.9277 

2.8951 

2.8661 

S.Otwi 

2.6187 

2.7955 

2.7763 

2.7587 

2.7486 

2.7278 

2.7141 

2.7014 

8.6896 

£.6060 

2.5252 

2.4478 

2.3719 


230.16 

19.296 

9.0135 

6.8560 

5.0503 

4.3874 

3.9715 

3.6875 

3.4817 

3.3258 

3.2039 

3.1059 

3.0854 

2.9582 

2.9013 

2.8524 

8.8100 

2.7729 

8.7401 

8.7109 

8.6848 

8.6613 

8.6400 

8.6807 

8.6030 

8.5868 

2.5719 

8.5581 

8.5454 

8.5336 

8.4495 

8.3683 

8.8900 

2.8141 



3.5806 

3.3738 

3.2172 

3.0946 

2.9961 

2.9153 

8.8477 

2.7905 

8.7413 

2.6987 

2.6613 

8.6883 

2.5990 


2.5727 
2.5491 
2.5277 
2.5082 
8. 4904 
2.4741 
2.4591 
2.4453 
2.4324 
2.4805 

2.3359 


3.5005 

3,2987 

3.1355 

3.0123 

2.9134 

2.8321 

2.7648 

8.7066 

2.6572 

2.6143 

2.5767 

2.5435 

2.5140 

2.4876 

2.4638 

2.4422 

2.4226 

2.4047 

2.3883 

2.3732 

2.3593 

2.3463 

2.3343 



3.7257 

3.4381 

3.2296 

3.0717 

2.9480 

2.8486 

2.7669 

2.6987 

2.6408 

2.5911 

2.5480 

2.5102 

2.4768 

2.4471 

2.4205 

2.3965 

2.3748 

2.3551 

2.3371 

2.3205 

2.3053 

2.2913 

2.2782 

8.2662 


2 

8.0164 

.9384 


840.54 

19.385 

8,8123 

5.9588 
4.7725 
4.0990 
3.6767 
3.3881 
3-1789 
3.0204 
2.8562 
2.7964 
2.7144 
2.6458 
2.5876 
2.5377 
2.4943 
2.4563 
2.4227 
2.3928 
2.3661 . 
2.3419 
2.3201 
2.3002 
2.2821 
2.2555 
2.2501 
2.2360 
2.2229 
2.2107 

2.1240 

2.0*01 

1.9588 
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TABLE C-4 (continued) 

r Distribution i Oppar 10 Paroant Point a 


Dtcraaa of F raatai in Itaaarator v 



l 

2 

3 

4 

5 

6 

7 

8 

9 

1 

39.864 

*9.500 

53.593 

55.833 

57.2*1 

58.204 

58.906 

59.*39 

59.858 

2 

8.5263 

9.0000 

9.1618 

9.8*3* 

9.8986 

9.3255 

9.3*91 

9.3660 

9.3805 

3 

5.5383 

5. *62* 

5.3906 

5.3*87 

5.309® 

5.8647 

5.8662 

5.2517 

5.2*00 

4 

4.5448 

4.32*6 

4.1906 

*.1073 

*.0506 

4.0096 

3.9790 

3.95*9 

3.9357 

5 

4.0604 

3.7797 

3.6195 

3.5802 

3-4530 

3.*045 

3.3679 

3.3393 

3.3163 

6 

3.7760 

3. *633 

3.2888 

3.1006 

3.1075 

3.05*6 

3.01*5 

8.9630 

2.9577 

7 

3.589* 

3.257* 

3-0741 

8.9605 

2.8833 

2.827* 

2.70*9 

8.7516 

2.72*7 

8 

3. *579 

3.1131 

2.9838 

2.006* 

2.7865 

2.6683 

2.6241 

2.5893 

2. 5612 

9 

3.3603 

3.0065 

8.8129 

a. 6927 

2.6106 

2.5509 

8.5053 

8.469* 

2.4*03 

10 

3.8850 

2.92*5 

2.7877 

8.6053 

2.5816 

2.4606 

2. 4i*0 

2.3772 

2.3473 

11 

3.2252 

2.859? 

2.6602 

8.5368 

2.4512 

2.3891 

2.3*16 

2.30*0 

2.2735 

12 

3.1765 

2.5068 

2.6055 

2. *301 

2.39*0 

C.3310 

2.2628 

2.2*46 

2.2135 

13 

3.1362 

2.7632 

8.5603 

2. *337 

8.3*67 

2.2830 

2.2341 

8.1953 

2.1638 

14 

3.1022 

2.7265 

2.5222 

2.39*7 

8.3069 

2.2426 

2.1931 

2.1539 

2.1220 

15 

3.0732 

2.6952 

2.4898 

2.361* 

2.273C 

2.2061 

2.1582 

2.1185 

7.0662 

16 

3.0*81 

2.6682 

fe.4€l8 

2.3387 

2.2*38 

8.1783 

2.1280 

2.0830 

2.0553 

17 

3.0262 

2.64 

2.4374 

8.3077 

2. £183 

2.1524 

2.1017 

2.0613 

2.0264 

18 

3.0070 

8.6239 

8.4160 

8.8658 

2.1958 

8.1296 

2.0785 

8.0379 

2.0047 

19 

2.9699 

2.6056 

2.3970 

2.8663 

2.1760 

2.109* 

2.0580 

2.0171 

1.9836 

20 

2.97*7 

8.5893 

8.3801 

2.2*89 

2.1582 

8.0913 

8.0397 

1.9985 

1.06*9 

21 

2.9609 

8.5746 

2.3649 

8.2333 

2.14<3 

8.0751 

2.06*38 

1.9819 

1.9480 

22 

2.9*86 

8.5613 

2.3512 

2.2193 

2,1279 

2.0605 

2.0064 

1,9668 

1.9327 

83 

8.937* 

2.5*93 

8.3387 

2. 2065 

2.11*9 

2.0*78 

1.59*9 

1.9531 

1.9189 

24 

8.9271 

2.5383 

2.387* 

2.19*9 

2.1030 

2.0351 

1 9826 

1.9*07 

1.9063 

85 

2.9*77 

8.5883 

2.3170 

2.18*3 

2.0982 

2.0241 

1..171* 

1.9298 

1.89*7 

26 

8.9091 

2.5191 

8.3075 

2.17*5 

2.0822 

8.0139 

1.9610 

1.9188 

1.38*1 

87 

2.9012 

8.5106 

8.8987 

2.1655 

2.0730 

8.0045 

1.9515 

1.9091 

1.87*3 

86 

8.8939 

2.5028 

8.8906 

2.1571 

2.00*5 

1.9959 

1.9*27 

1.9001 

1.8652 

89 

2.8871 

8.4955 

8.2631 

8.1*9* 

2.0566 

1.9878 

1.93*5 

1.8918 

1.8560 

30 

2.6807 

2.4887 

2.8761 

2.1*82 

2.0*98 

1.9803 

1.0269 

i.?8*l 

1.8*90 

40 

8.835* 

2.4404 

2.2261 

8.0909 

1 9968 

1.9869 

I . QU 5 

1.8289 

1.7929 

60 

2 .791* 

2.3938 

2.177* 

2.0*10 

1.9*57 

1.8747 

1.8194 

1.77*8 

:.7j8o 

120 

8.7*78 

8.3*73 

2.1300 

1.9983 

1.8959 

1.8236 

1.7675 

1.7220 

i.e?*3 

m 

2.705* 

2.3026 

8.0838 

1.9**9 

1.8*73 

1.7741 

1.7167 

1.6702 

1.6315 
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TABLE C-5 

CRITICAL VALUES dct(n) 01 THE MAXIMUM ABSOLUTE 
DIFFERENCE BETWEEN SAMPI2 AND POPULATION 
RELIABILITY FUNCTIONS 


Sample 

Level of Significance (a) 

mISV 

00 

0.20 

rwm 

0.10 

0.05 

0.01 

3 

0.565 

0.597 

0.642 

0.708 

0.828 

4 

0.494 

0.525 

0.564 

0.624 

0.733 

5 

0.446 

0.474 

0.474 

0.565 

0.669 

10 

0.322 

0.342 

O .368 

0.410 

0.490 

15 

0.266 

0.283 

0.304 

0.338 

0.404 

20 

0.231 

0.246 

0.264 

0.294 

0.356 

25 

0.21 

0.22 

0.24 

0.27 

0.32 

30 

0.15 

0.20 

0.22 

0.24 

0.29 

35 

0.18 

0.19 

0.21 

0.23 

0.27 

4o 

C .17 

I 0.18 

0.19 

0.21 

0.25 

45 

0.16 

0.17 

0 . 1 & 

1 0.20 

| 0.24 

50 

0.15 

0.16 

0.17 

0.19 

0.23 

Over ^ 

1.01 

1.14 

1.22 



50 J 

VPT 

* 

< ffI \ 

! yir 

& 










oooco ooooo 
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TABLE C-6 (continued) 


Reliability Exponential Function (R - •"**) (Cont.) 



it 

R 

It 

R 

1 

0,110 

o.iao 

* 

©.65 

0.70 

0.522346 

O.496585 

1 

0.130 

0.100 


c.80 

0.472357 

a 4*9329 

0.427415 

1 

0.150 

0.860708 

0.85 


0.160 

0.170 

0.852144 

0.843665 

0.90 

0.95 

0.406570 

0.3867*1 

0.367879 

O.349938 


0.180 

0.190 

0.§35i70 

0.82*959 

1.00 

'..05 


0.200 

0.818731 

1.10 

0.33287* 


0.210 

0.6_0«S4 

1.15 

0.316637 


0.220 

0.802519 

1.20 

0 30119* 


0.230 


1 25 

a 286505 


0.240 

1.30 

0. 272532 

0 259240 


0.250 

0.7^8801 

1-35 


0.260 

0.771052 

1.40 

0.246597 


0.270 

0.280 

0.290 

0.763379 

1.45 

1.50 

l:& 

0.234570 

0.223130 

0.212248 


0.300 

0.740816 

c. 201897 

' 

0.310 

0.320 

0.733447 

0.726149 

}-x° 

1.80 

0.182684 

0.165299 

0.149569 

i 

0.330 

0.340 

0.350 


1.90 

! 

1M 

2.00 

2.10 

0.135335 

0.122456 


O.36O 

0.697676 

2.20 

0.110803 

j 

| 

S.iiS 

©.■*90 

0.677057 

l:tt 

2.50 

0.100259 

0.090718 

O.O82085 

i 

0.400 

0.670320 

2.60 

0.07427* 

i 

J 

0.410 

0.663650 

0.657047 

2.T0 

0.06^2-6 

1 

C. 420 

2.80 

0.060810 

i 

i 


£18$ 

2.90 

3.00 

a 055023 

aotv ?8r 
0.038774 

! 

0.450 

0.63762C 

3-25 

« 

0.460 

0.631284 

3.50 

0.030 1 97 


0.4t0 

Q.ibo 

0.6250® 
a 618783 
o. 612626 

3.75 

4.00 

0.0i^51b 

C. 0’4^io 


0.490 

4.25 

0.014264 


0.500 

c. 606531 

4.50 

0.011109 


o.sn 

0.600496 

4.75 

0.008652 


0.520 

& 5^4521 

5.00 

O.OO 6738 
a 00408? 



& JomOJ 

5-50 

j 

0.5*0 

0.582748 

6.00 

0.002479 


0.550 

0.576950 

1 6 ‘ 50 

0.001 503 

t 

0.560 

0.571209 

7.0® 

0.000912 



0.5655*5 

7.5© 

a 000553 



Ch 

8.00 

0.000335 


0.59? 

G. 55*3^7 

1 9.0C 

0.000:23 ; 


G.fio 

0.5*8812 

1 10.0 

0000^5 1 

1 


* c; 

*• A. ? 












a**cp 7oe-iai 


b 



\ 

i 




i 

i 

i 


a 


S3 



5 


To Use: Multiply value shown by total part hours to get M?»F figures ixi v ours 

Note: <J=2r, except fox the lower Unit ca tests truncated at a fixed tine and where r<a. to such oases 
d*2<r+l). 
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TABLE C-8 



I REQUIRED NUMBER OP FAILURES FOR VARIOUS VALUES 

| OF CONFIDENCE AND PRECISION 

| ( Exponential Distribution) 

Precision - 6 

Confidence 

85# 

90# 

95# 

wm. 

5# 

830 

1082 

1537 

2655 

10# 

207 

271 

384 

664 

15# 

92 

120 

171 

235 

20# 

52 

6? 

96 

166 

25# 

33 

43 

61 

106 

30# 

23 

30 

43 

74 

| 35# 

17 

22 

31 

54 


Example: 43 failures are required to be 90# 
confident that the estimated MTBF is within 25# of 
the true value. 
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TABLE C-9 
OAMKA FUNCTION 


I 

Values of r(n) - t m e~ x x”" 1 dx; r(n + 1) - rT(n) 


C 


n 

■B9I 

n 

r(n) 

n 

r(n) 

n 

r.(n) 

1.00 


1.25 

0,90640 

1.50 

m 

Em 

0.91906 

■SI 

0.99433 

1.26 

0.90440 

1.51 

HI 

fia 

0.92137 
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APPER8IX D 

6 m FOR REYKWElS OF STONES C0NTAHN6 
COST-EFFECTIVENESS ANALYSIS * 

CHAPTER I 

GENERAL BACKGROUND 


jgroductton 

To assist to the review of studies containing cost-effectiveness analyses, 
a series of key questions with explanatory notea have been prepared and are 
contained in the nexs chapter. These queatiooa, taken together, will nut 
necessarily cover all aspects of all cost -effectiveness analyaee. No one 
general liat of quest! ms can do that. Rather, die questions are designed 
to focus the attention of the reviewer on selected aspects ro assist him in 
evaluating the analysis. All the questions are not' applicable to all studies 
and they are not necessarily of equal Importance to those studies where they 
do apply. The reviewer must exercise his judgment on whether the questions 
are applicable and the oegree of applicability tc the study being reviewed. 
These questions that are considered particularly important and of widest 
application have been underlined, and are also listed separately for conven- 
ience in the beck as "SELECTED QUESTIONS."** This document is intended 
only as a guide and not as a full and comprehensive treatment of all aspects- 
of cost-effectiveness analysis. 

Questions that do not beer on military cost -off ectiveneso analyses are 
not included in the next chapter. Furthermore, no questions are addressed 
to the subject of the intuitive Judgment and other factors used in making 
decisions to which cost-effectiveness analyses contribute. 

Cost-Effectiveness Analysis and the Estimate of the Situation 

Ciet -effectiveness analysis is a method far studying how to make the 
best of several choices. By cost-effectiveness is meant the relation of the 
resources required (cost) to achieve a certain ability to accomplish an 
objective (effectiveness). The term cost-effectiveness is always used in 
relation to the effectiveness of alternative systems, organizations, or 
activities. 

Cost-effectiveness analysis is based on the economic concept that all 
military decisions involve die allocation (best use) of limited resources 
among competing requirements. The allocation is determined by studying 
how to get the best use of the available resources. This svme concept is 
embodied in Army decision processes, ft is used by a combat commander 
when he determines (estimate of the situation) the allocate - of his resources 


Questions l, 4, 12, IS, M, 22, 23, 31, 37, 44, 46, 57, §0, 66. and 73. 
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(forces) among the mala and secondary efforts and r es er ves In the offense 
or between the forward and reserve forces in the defense. A 03 uses the 
same concept In preparing his recommendations for res! locating amrag the 
dements of the command the ammunition available stggdy rats aanoonesd 
by the higher headquarters. The company commander goes throtgh the same 
process in deciding bow to spend his company funda. 

Although cost -eff ecti venese analysis and the estimate of the sH a tthn 
are similar in concept, they differ In several as pec ts. The purpose of 
m estimate of the sttuatian is to arrive at a re comm e n ded course of action 
ft la usually a process to arrive at decisions to solve "today's problems 
today." It does not concent Itself, in s realistic sense, with problems, 
operations, or systems of the future, evsn though it is sometimes not 
clear where the problem of tcday ends and the problem of tomorrow starts. 
Because it deals with relsiivtly immedtots problems, the formulation of 
possible courses of action In an estimate of the situation is severely limited. 
The resources (force* and weapons) available to the commander are fixed 
by what has been made available and there is no real flexibility in changing 
their composition or baste organisation, b practice, it Is also usually 
difficult to obtain additional resources from die next higher commander. 

Another severe constraint on die estimate of the sttuetlon la die time 
factor. • ^formation is usually incomplete and the time avnilmbla tefare a 
decision la required often does not permit filling In gaps — even If it were 
possible. Often there ts only sufficient time to analyse the mission, gather 
staff estimates, formulate a few possible coo-see of action and quickly 
weigh these courses of action against the en e m y cr a b t lfHee for diffi c ul tie s 
to be overcome) and with each other, and select a course of action based on 
some criteria — often called the governing factors. Time usually does not 
permit testing the range of the dependence of the proposed course of action 
on the staff estimates and planning assumptions. 

Military cost-effectiveness analysis ic not a decision process but an 
aid in facilitating decisions that must be made now in regard to development, 
force composition, and logistical and manpower policy pr o b lem s in order 
to be prepared for wars in the future. The analytical techniques employed 
in cost-effectiveness analysis are required to stgipUment th o s e employed 
in the estimate of the situation because, as we look into the future, the 
number of uncertainties multiply. These imcertalulles Include such thingr 
as planning factors, the enemy and his reactions, the strategic c oe o e pt . 
technology, change, and even the national objectives which can be expected 
to change in the future as alliances shift and new forces in the world devslojc 
Advances in technology crci.e new opportunities that may require changes 
In organ Untie n and doctrire as well as hardware. All these uncertainties 
lead to a large number of variable# that must be considered. Some of t h ese 
variables are subject to our control, some to the enemy's and others to 
nobody's control. But all are variables, and all are int erd ependent. 
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The tnereaee la somber* sad kinds of variables associated with 
problems of tte Mo re mb be fUoetrated, oa a small scale, in a hypothetical 
a&uty of a Adore weapons system for an infantry platoon— assuming that the 
fefentry platoon wUl be presmt hi the time frame tader consideration. Tbs 
variables that weald reqvire stnty would include each parameters as alter- 
n at ive w a apwe sy stems that oaa be available in the time frame under study, 
com po siti on pntx) of kinds of weapons w ithin the total system, tbs number of 
tadtvidaal weapo ns within nch mix of weapons, levels of warfare, expected 
leeales of war, and offsets of m pp or tin g weapons of higher echelons, if 
each of th o se p e rim e ters takes only three alternative velaee, for example, 
levels of warfare to be considered are n u clear warfare, conventional war, 
and one type of stability of operations, over 700 eases result— and all 
signjftoaid parameters have not been listed, ff tte number of candidate 
weapons systems is increased from three to six there are over 1400 case# 
to be considered. 

1 is in ibis environment ot uncertainties and flexibility in use and inter- 
changeability of reeooroee freople, dollars, and hardware) that ooet-effective- 
ossa analysis is a useful aid. 1 assists in providing increased insight into 
the problem and ae modi relevant information as possible in order that the 
decision maker oaa oanosntrato an those areas where Judgment must be 
applied, particularly in consideration of qualitative aspects and consistency 
Witt higher echelon considerations. For example. In a hypothetical force 
composition problem where flexibility in force composition is possible. It 
bee been determined that tbs force must have a capability to destroy certain 
kinds of burguta at certain expected ranges. Two possible alten*Uves are 
artillery and t ao ti oal air. The time required and the cost to destroy these 
targets by use of each alternative can be calculated However, the importance 
of having a capability to attack these targets at any time of the day cr nigk, 
regarweas of w ea t h er oo n d ttt o n s , la a matter of judgment. This judgment 
can be better made when the cost-effectiveness of each alternative is known, 
in otter words, tbs price to be paid for an aU -weather capability stated in 
sufflcisot detail and accuracy to be useful for plaaniag. 

The effort to provide information so the commander can bettor exercise 
his judgment is also found in tbs estimate of tte situation process. For 
ex a m ple , a oombat commands? can better apply his judgment to selection 
among possible course? o i actions when be hnt staff antes— even if 

only rough-of the matter of casual ties be will suffer and the time required 
to accomplish the mission for each of the proposed courses of action. 

However, the variables that a staff sail mats must contend with arc relatival? 
limited. The friendly organisation is fiW, there is only one enemy in only 
one area and tte optic u s opes to the seamy are relatively few. For example, 
tte enemy can attack, defend, or execute some variation of a withdrawal. 

For practical purposes, neither the enemy nor the friendly force mm introduce 
nsw weapons systems or change then fu»Wa mental organisation or doctrines 
la tte time period covered by tte eettnwte aFtt* ittuntkm. 
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The buU of cost-effectiveness analysis U that there are alternate way# 
of reaching an objective and each alternative requires certain recourses and 
produces certain results. This is the same basis of the estimate of the 
situation which studies proposed courses of actions* each of which requires 
certain resources (forces and supplies) and pratoes oertaia expected results 
(time to take the objective, casua lt i e s incurred). A cost-off ectivene j» 
analysts is designed to examine systematically and relate ooets, effectiveness, 
snd risks of alternative ways of accomplishing an o bj ect i ve and designing 
additional alternatives (proposed courses of action) If those examined are 
found wanting, t is an analysis of the cost gad effectiveness of a system* 
such as a forward area air defense or sn air mobile division, and all of the 
system implications, ft can be considered as a kind of Consumers Research 
to as el at in getting the most for the resources to he expended and not as a 
•e&i ch for the cheapect regardless of effectiveness. 

A major methodological difference between cost -effectiveness analyses 
end ether military studies is the manner in which the results ere presented. 

A staff study or a staff estimate, tike a cost-effectiveness analysis, considers 
sources of action (alternatives). However, the staff estimate and staff 
study usually embody a single recommendation with the other alternatives 
either rarsly mentioned or not as fully discussed as toe recommended course 
of action. The commander (decision maker) la given the full reasoning behind 
the recommended course of action which to frequently presented so that the 
only option open to a ’'yea" or "no" decision. 

In a cost-effectiveness analytic, the significant alternatives, the 
available facto, the reasoning process and the p e rti ne nt considerations per- 
taining to each significant alternative are presented. All identiflaNe 
assumptions and data are presented so that thotr validity can be questioned, 
hi addition, snd this to a major goal of a cost -effectiveness analysis, the 
dependence of the results of the analyses on these assumptions snd data 
are tested. 

The staff estimate and staff study do identify major assumptions. 

However, an Implied assumption to often introduced when several different 
courses of action are open and a decision to mads to proceed in one direction. 
SUch a decision la then accepted as a known quantity when, in reality, it 
really Is an assumption. There are many reasons for such assumptions, 
but frequently the result of ths study or estimate is not tec ted for sensitivity 
to such hidden assumptions. 

Cost- Tectiveness analysts places great emphasis on use of numbers 
and calculations in any effort to determine quantitative factors where possible. 
Of cmirse, there. a;'« many aspects at military activities that cannot be 
reduced to a quantitative factor. There la now no valid way of assigning a 
number to morale, the psychological effects of s certain military operations, 
or a hc^i of other (actors. However, it is possible to calculate the number 
of l&S-mjB. howiuer rounds end fetal cost required to destroy a certain type 
of tanret. The impact of (actors such as morale, training, reliability of 
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allies cannot yet be c alculated and ere now matters of Intuitive judgment. 

A cost-effectiveness amu'/als seeks to quantify what can be logically 
calculated so that the decision maker knows the extent to which intuitive 
judgment must be used in making a decision. 

Essential Element s 

Hie essential elements of a cost -effectiveness analysis are: 

1. Objective (s) (functions to be accomplished). 

2. Alternatives (feasible ways of achieving 'he 
desired military capability or accomplishing 
the function). 

3. Coat of resources required by each alternative 

4. A set of mathematical or logical relationship*; 
among the objectives, alternatives, environment 
and resources (models). 

5. A criterion for choosing the preferred alternative. 

The Objective 

The determination of the objective U often complex, ft> order to design 
alternatives properly, the problem must lie analysed to determine the real 
fractional need underlying the requirements for certain organizations and 
hardware systems. Thorough examination of the functional need usually 
brings insight into the problem and leads to generating alternatives that 
may accomplish the desired goal. Close examination of objectives stated 
only in terms of specific organizations or systems often discloses that the 
net result is not a significantly new or improved capability bid a relatively 
miner product improvement. This does not imply that product improvements 
are not needed but rathor that a full understanding of the true significance 
cf what is being proposed for purchase is necessary. Far example, in 
stating a requirement for an artillery system with a specified minimum range 
capability, the real objective may be a capability to destroy certain kinds 
of target* under certain conditions. By examining the problem from the 
functional basis, the planner is better able to understand the problem. This 
Insight may lead to other alternatives that should be studied. The examina- 
tion may ahow that the proposed new aniilery system is only one alternative 
m accomplishing the objective and that another alternative may be preferable. 

There are practical limits on the definition of the objective. Every 
military activity is part of a larger activity anc it is necessary to draw the 
ltnss at some point. However, th« objective should not be unduly restricted 
by confusion with performance character is* ics such a^ speeds, weights, 
muzzle velocities, hit-kiU prebabi* Mes, anu so forth. 
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la military planning there is rarely only one exclusive way of achieving 
a given objective. Each way has its own price tag of time, men, facilities, 
materiel, and money . Assume, for example, that the planning problem — 
admittedly over -simplified — is to design a new type of division with certain 
capabilities, hi satisfying these capabilities, the TOE designer has many 
alternatives. For the same capability, is it better to have more mobility 
(trucks, aircraft, and other vehicles) and less manpower, or perhaps more 
mortals and fewer riflemen? The alternatives are limited only by creative 
imagination and good military judgment. By exploring alternative ways of 
using resources it is often possible to discover ways of achieving an ob- 
jective with fewer resources, or accomplishing more with the same resources. 
All feasible and significant capabilities to accomplish the objective should 
be considered, including the capabilities of the Navy, Air Force, and Marines. 
Prejudices, "party-line" and other Vms of preconceived notions should be 
avoided in the design of alternatives. 

Coat 


Determining the cost of each alternative is based cm incremental costs. 
These are the net co3ts of adopting the alternative. Such costs are deter- 
mined after due allowances for those resources already paid for regardless 
of whether the alternative is adopted, ?nd would be available for use under 
the alternative if it were adopted. In determining the cost of an alternative 
all the resource implications are considered. The alternative is treated in 
a system context. For example, the cost, admittedly oversimplified, of 
adopting a new radio would include not only the cost of the radio and its 
development, but also the costs of training people to operate it, the total 
cost of maintaining the radios, and the cost of the additional radios required 
for maintenance float, replacement, combat consumption, and so forth. 

Costs need not be stated in precise terms down to the last dollar or man. 
However, the coats must be accurate enough to permit evaluating ihe 
military worth (effectiveness) together with the costs. Like everything else, 
this rule must be applied with discretion, hi dealing with systems way out 
in the future die accuracy of the ost estimate, whether it is an absolute 
figure or a range, probably is inverse to the distance out in the future. 
Usually cost estimates are tested by sensitivity analysis. These are re- 
petitive analyses using different quantitative values to determine if the 
re3ulis are sensitive to the values assigned. Such analyses give the 
decision maker a better understanding of hoy/ much uncertainty is involved 
if the . e are significant errors in the cost estimates. He can then better 
judfre if the investment is worth the payoff considering the uncertainties 
involved. 

Models 

Model? are used in cost-effectiveness analysis to cope with the host of 
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variables that are inherent in problems of the future. A model is simply 
certain relationships expressed in some way tc simulate real or expected 
conditions in order to foresee, even to s limited extent, the expected oii- 
oome of a course of action. Models assist in simplifying the problem, in 
identifying the significant components asd Interrelations, in determining 
which variables are especially important for die decision at issue, and which 
variables can be suppressed. & this manner, the decision process can be 
more precisely focussed on those areas which require a judgment decision. 

Models range from simple graphs to complex equations and can also take 
fim form of a wargame or field maneuver. The estimate of the situation and 
staff estimates also use models. The comparison of proposed courses of 
action against enemy capabilities or expected difficulties and the comparison 
among the proposed courses of action represent uses of models to foresee the 
future outcome of an action. 

All models, simple or complex, ar» abstractions of the real world and 
their validity depends on the proper selection of assumptions and the 
correctness of the relations portrayed, and file pertinence of the factors 
included in the model. Two aspects of model building are particularly 
troublesome, quantification and the treatment of uncertainty. Some variables 
are difficult to quantity, such as the continued availability of ceitain support 
from an ally. This difficulty leads either to the neglect of such variables 
by ignoring them or by a qualitative modification of a solution derived from 
the treatment of other . ariables that have been properly quantified. Such 
treatment often results in the diffient-to -quantify variables being lost within 
all the other qualitative considerations that must be weighed when the time 
comes to recommend action on the basis of the solution from the model. 

The influence of the variable that cannot be quantified and all uncertain- 
ties must be specifically addressed in the model unless it can be demonstrated 
by logic or analysis that they are trivial, affect all alternatives roughly the 
same, or the results are insensitive to them. Guessing may lead to disaster. 
For example, if there is uncertainty about 8 factors, a best guess might be 
made on each of them. If there is a 60% probability that each best guess is 
right, then the probability that all guesses are right is less than 2%. Relying 
on t ;st guesses, in this case, would be ignoring all the outcomes with more 
than 98% probability of occurring. Uncertainties and the problem of th* 
factors that cannot be quantified can be handled through various techniques 
such as Monte Carlo sampling, contingency analysis, (see Glossary) and 
even wargaming for certain purposes. 

Models that portray relations incorrectly also lead to false results. 

For example, some models are based on the persistence principle which 
states that what is happening or has happened will persist, T his type of 
model is dangerous except for very short-term uses. For example, .it is 
wrong to assume that the enemy tactics used during the Kci-ean war will 
continue to be used in the future against new types of equipment and tactics 
that may be introduced. Seme models depend on extrapolation which assumes 
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that trends will continue uninterrupted. 8uch models lend themselves 
readily to mathematical treatment but are often erroneous because of 
failure to connlder what is called the Law of Diminishing Returns. For 
example, a machine gun can fire at a certain high rate. However, this 
high rate cannot be maintained for vary long (extrapolation) because the 
barrel would soon be burned out. 

Models can be classified into two general types— enct (deterministic) 
or probabilistic. An exact model of warfare, of course, is impossible in 
peacetime. However, it is possible to create an almost exact model of 
some specific piece of hardware or activity and subject it to test. The 
final product of the model will then closely approximate the results from 
the actual hardware or activity. March graphs used for planning adminis- 
trative movements are examples of deterministic models. Most military 
problems are, by nature, made up of uncertainties. Consequently, they 
are considered as probabilistic when the uncertainty is identified by a 
probability factor. For example, a war game using a certain kill probability 
for an air defense system is a probabilistic model. 

The construction of models to evaluate effectiveness is often difficult. 
The difficulties arise in selection of criteria of effectiveness. It is 
relatively easy to measure the comparative effectiveness of two similar 
pieces of equipment designed to accomplish the same general objective as, 
for example, in comparison of a towed i05-mm and a self-propelled 105-mm 
howitzer. However, it is more difficult to compare the effectiveness of 
general purpose force organizations iuch as two different kinds of divisions 
or even two equal-strength infantry battalions having the same general kinds 
of weapons but (me having three rifle companies and the other having four. 
The impact on effectiveness of intangibles such as morale and leadership 
can hardly be calculated and requires the application of judgment. Each 
study virtually requires a consideration of its own criteria of effectiveness. 

Models used in cost-effectiveness analysis sometimes tend to become 
mathematical and abstract. Consequently, they may be difficult to under- 
stand. A good cost-effectiveness analysis strikes a balance in the use of 
models between simplicity and retention of enough detail to ensure that 
the expected outcome of an expected action will be adequately portrayed. 

In any case all models have certain common elements. These are broadly 
stated as a definition of toe problem,princlpal factors or constraints, 
verification and toe decision process — or application of criteria. The 
validity of conceptual or mathematical models cannot be verified in a cost- 
effectiveness analysis by controlled experiments. At the best, they can be 
tested by their workability. Questions 37 to 44 in the next chapter are 
designed to assist a review to test the workability of models used in cost- 
effectiveness analyses. 

Criteria 

The most widely used criteria in Army studies for selecting the pre- 
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furred alternative are usually baaed on either equal coat or equal effective- 
ness of the alternatives. Another method, known as incremental effectiveness 
at incremental cost, is used in special cases, ha die t,ual cost form it 
is assumed that there is an arbitrary fixe d budget or series of fixed budgets, 
and the analysis determines which alternative gives the greatest effectiveness 
for the same expenditures or resources, hi the equal effectiveness form, a 
specified aud measurable military effectiveness (capability) is stated and 
the analysis is to determine which alternative achieves this effectiveness 
at least cost. The incremental effectiveness at incremental coat method 
relates the increase in effectiveness achieved to the associated increase in 
resources involved. This method is normally uced only as a bet resort 
when neither costs nor effectiveness of alternatives can be made equal, e. g. , 
when a capability based on a new technology is to be added to the force and 
this new cspablltty cannot be approximated by any practicable combination 
of existing materiel and men. 

Role of Judgment 

Judgment is used throughout a cost-effectiveness analysis in the same 
manner as in the making of an estimate of the situation or a staff estimate. 
Judgment is used in analyzing the objective, deciding which alternatives 
(courses of action) to consider, which factors are relevant sod the inter- 
relations among these factors, which numerical values are to be used, and 
in analyzing and interpreting the results of the analysis. The goal of a 
cost-effectiveness analysis is to keep all judgments in plain view and *o 
make clear the logic used. It also shows the sensitivity of the results to 
the significant judgments made. The depth of a cost-effectiveness analysis 
is tempered by the time and manpower available and the importance of the 
subject matter. A cost-effectiveness analysis requires resources and it 
must serve as an aid to the making of decisions and not be a mere Intellectual 
exercise. 

Review of Studies 


There are probably almost as many different ways of reviewing a study 
containing cost-effectiveness analysis as there are reviewers. Furthermore, 
the time available for review is variable and studies lack a common format. 

It Is suggested that the points listed below be checked specifically in the 
early stages of a review. 

a. Statement of criteria used to judge effectiveness. 

b. Statement of criterion used to select preferred 
alternative. 

c. Use of incremental costs. 

d. Explanation of logic of models. 
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e. Presence or lack of analysis of seniftivtty of the 
results to significant data and assumptions. 

Without these elements being present, the study ’/ill probably be 
of poor quality. 

Army— conducted studies containing cost-effectiveness analysis 
usually do not have a uniform organisational pattern but many generally 
follow the Staff Study format. On that basis, the key questions in the next 
chapter have been grouped under these headings: 

Statement cf the Problem 

Assumptions 

Alternatives 

Documentation 

Coet 

Relationships (Models) 

Effectiveness 

Criteria 

Conclusions and Recommendations 

The grouping unde- the above headings inevitably leads to some dupli- 
cation of material, particularly on the use of analytical tools such as sen- 
sitivity and contingency analysis. This duplication has been kept to a 
minimum but full coverage has been retained under each heading as a 
convenience to the reviewer who wishes to refer to a specific heading. 

The Glossary is designed to give a non-technical definition of terms 
frequently used in cost-effectiveness analyses. 

The annotated Bibliography has been designed for the reviewer who 
desires to read further into the methodology of coat- effectiveness analysis. 
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1. IS THE PROBLEM STATED THE REAL PROBLEM ? 

An improper statement of the problem often results in either studying 
the wrong problem or precluding consideration of worthy alternatives. 
These defects are usually avoided by a statement o 2 the problem in terms 
of a functional need— the Jcb(p) to be done— without implying how it is to be 
done. A statement of the problem in terms of requirements for kinds of 
forces, systems, or performance characteristics, except if it is a 
follow-on to a previously approved study of a functional need, should be 
critically examined to ensure that the wrong problem is not being studied 
and that worthy alternatives are not automatically excluded from 
consideration. For example, although the stated problem (no previous 
study of functional need) may be to select a rifle to meet certain capabili- 
ties (requirement statement) , the real problem might be providing the 
rifle squad with adequate firepower to accomplish certain functions (func- 
tional need). In such a case, a rifle is only one possible alternative. 

A word of caution is in order. There often is a practical limit on the 
depth of the statement of the functional need or the study may become 
unmanageable. For exar pie, in the case cited the functional need could 
be conceivably so stated that the rifle squad itself becomes only one altern- 
ative to solving a larger problem. To avoid this difficulty, either certain 
broader decisions must be considered as made, thereby narrowing the 
scope of the study, or the broader study undertaken. When the former 
approach is taken, the study is known es a suboptimization. The reviewer, 
based on his knowledge and judgment, must determine if the suboptimiza- 
tion has so narrowed the scope of the problem that the real problem has 
been missed or worthwhile alternatives excluded. 


2. DOES 1HE STUDY IDENTIFY IMPLIED SIGNIFICANT COMPONENTS 
OF THE PROBLEM THAT MUST BE FULLY TREATED IN THE 
STUDY? 

Like the mission statement In an estimate of the situation, the problem 
to be treated in a cost-effectiveness analysis must be analyzed to identify 
all functions that must be performed Some of these implied functions are 


•Those questions that are considered particularly important and of widest 
application have been underlined. 
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n <* anoarent at first. The reviewer should watch for implied signifi- 
cant component parts of the problem that are neither identified nor treated 
fully in the study. The reviewer should also watch for other problems 
thafare opened up or revealed by the study that should be further investi- 
gated. 


o 
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ASSUMPTIONS 


3. ARK ALL ASSUMPTIONS IDENTIFIED? 

The reviewer should watch (or assumptions that are not identified as 
such because assumptions imply a limitation or a Judgment. In order to 
evaluate the study properly, it is necessary to assess the impact of the 
limitations and the validity of the Judgments contained in all the assump- 
tions. An example of a common assumption that is often not identified is 
that a given unit operates by itself. As a result, in measuring the 
effectiveness of a division, for example, inadequate co-sidentton is 
sometimes given to the support the division receives from non-dlvisional 
units such as corps artillery or tactical air unit*. This failure to consider 
non-divisional support may lead to erroneous conclusions and recommen- 
dations. Another frequently hidden assumption is that the enemy' s doctrine 
and tactics are rigid although ours are flexible. 


4. ARE THE ASSUMPTIONS UNDULY RESTRICTIVE i 

Assumptions are properly used to narrow the scope of the study to 
manageable proportions. However, the assumptions should be examined 
to determine whether they unduly restrict the study by eliminating possible 
significant alternatives or by narrowing the scope of consideration to the 
point that the conclusions and recommendations may be in error. This 
examination may be required throughout the review of the study and not 
only during the review of the stated assumptions. 

Assumptions covering the subjects listed below often unduly restrict 
the scope of the study and lead to questionable conclusions and recommen - 
dations. 


a. Non-availability or limited availability of 
support from otter services (e.g. , 
tactical air support or MATS effort) . 

b. Locale of operations. 

c. Duration and intensity of operations. 

d. Enemy organization, operations, and 
reactions to our decisions. 

e. Time period covered. 
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5. DO ANY Or THE MAJOR ASSUMPTIONS UNJUSTIFIABLY TREAT 
QUANTITATIVE UNCERTAINTIES AS TACTS? 

An uncertainly can be defined aa the lack of definitive knowledge for 
assigning values or probabilities to factors that influence decisions. 
Uncertainties can be either quantitative (risks) or qaalitativ*. (See 
UNCERTAINTY and RISK in Clossary.) Examples of qua ntit a ti ve uncertain- 
ties arc hit-kill probabilities, equipment availability rates, ammunition 
expenditure rates, and reliability statements. Tbs availability of base 
rights in a foreign country at some Altars Ums, or tbs start of aggression 
by the potential enemy in a given year arc example* of a qualitative un- 
certainty. (See next question.) 

The reviewer should be alert for st at ed and implied major assump- 
tions that assign fixed values to quantitative uncertainties and then treat 
these estimates as tecta. A common •xampla is the assumption that s 
proposed weapon system will ha vs a certain hit-kill probability, ft is often 
better to handle significant uncertainties by oensitivity analysis. This is 
s repetitive analysis using different quantitative values to determine if the 
result? are sensitive to the values assigned. When significant uncertain- 
ties are treated as tecta by assumption, tbs conclusions and recommenda- 
ti uus of the study may be no more valid than the assumption unless it can 
be demonstrated that the conclusions and recommendations are not 
sensitive to plausible errors in tbs " tecta. " 

The number of sensitivity analyses required, and feasible, is a 
matter of judgment. There are limits to the time and manpower available 
for a given study. Sometimes aa educated guess, considering all the 
circumstances, will suffice. In effect, the reviewer must judge whether 
the study agency has performed ad equa t e sensitivity analyses considering 
all the circumstances, the importance of the subject, and whether further 
sensitivity analysis may significantly affect the conclusions and 
recommendations . 


8. DO ANY OF THE MAJOR ASSUMPTIONS TREAT QUALITATIVE 
UNCERTAINTIES A8 FACTS? 

Major qualitative uncertainties trsated as assumptions also tend to 
dictate conclusions. A common qualitative uncertainty that may dictate 
the conclusions concerns the estimate of the enemy. Many studies are 
based on intelligence estimates, or target arrays prepared or approved 
by the Defense Intelligence Agency. Hear ever, these estimates are some- 
times assumed to be tecta. In such cases, this often results in the enemy 
being considered to be inflexible and no allowances are made for him to 
react ic different ways to our operations or to uur Introduction of new 
capabilities. When it ie not definitely known how we will operate or be 
equipped 10 yeers hence, it ie queeUon&hla to assume that the eosr.y 
operations and equipment In the future can be predicted with certainty > 
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Other qualitative uacertainties, stated oar implied, that should be 
totaled with caatlcn in tbone associated with political cootfaSentlou. 
Iwaptes in availability of bass rights, assurance o f overflight per- 
mission, sad composition of political sad military alliances <m either side. 

Treatment o 4 the kinds of uoeertaiaties discussed above la aa analysis 
Is not simple, bat the effects of seek uncertainties on the conclusions should 
not be asgjscted hi a study. One method to oops with signifies* uncertain- 
ties of this kind is to use oontingsncy analysis. This involves repetitive 
analysis with different initiative assumptions, each as type at conflict or 
ene my capabilities, to determine their effects (or comparison with tbs 
results of the initial analysis. The amount of coatlagercy analysis required 
has to be a matter of jud gm e n t, as discussed in tbs previous question. 


7. ARE THE MAJOR ASSUMPTIONS REASONABLE? 

Major assumptions should also be tested to determine if they are 
reasonable. This test can be facilitated if the ready documents or provides 
some explanation of why snob assumption was made so that the reasons can 
be evaluated by the reviewer. A useful technique for reviewers is to try 
to thfek of other major assumptions that are plausible, if these invalidate 
the conclusions sad recommendations, then the study is cpestlooeble. 
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ALTERNATIVES 


8. ARE CURRENT CAPABILITIES ADEQUATELY CONBDERED AMONG 
THE ALTERNATIVES? 

Current capebilittee should sot be omittvd from consideration lx 
construction of alteruativsa except for valid nuom that are dearly 
stated. Wild reasons nay taoluds Suture of the war se t system to aooom- 
pliah the current mission, or a significant d e gr ad ation of eepabiltty relative 
to that of the potential enemy. Consideration of onrrent capabilities ts as 
improvement that Is worth the exp enditur e of asm reeooroee. By cooaider- 
iaf current capabilities, much of whoee coats are already paid for, aa aa 
alternative, the studr can show the diffsrenoe la sffeotlveaees and coeu 
that result from the adoption at the pro p osed new system or organisation. 
(See question 33.) Current capabilities should also he considered, where 
appropriate, a component of aa alternative. 


9. ARE '* TRADE -OF 1 8" WITH EXISTING SYSTEMS OR ORGANIZATIONS 
ADEQUATELY CONSIDERED WITHIN THE ALTERNATIVES? 

Where appropriate, the design of alternattvee should cue rider 
" trade-off*" with existing systems or oar gaol rations. Possible examples 
are: (i) in siudyifii thv Increased use at Army transport aircraft an 
alternative might include reduction in other mean r of transport; (2, in a 
study on an Improved fire control system sn alternative might Include a 
reduction in ammunition stocks (e. 


10. ARE THE APPROPRIATE CAPABILITIES OF THE AIR FORCE, NAVY, 

OR MARINE CORPS CONSIDERED AMONG THE ALTERNATIVES? 

The alternatives should consider the capabilities of Air Force, Navy, 
or Marine Corps as appropriate. The Army usually conducts combat 
operations with the support of one or more of the other Services and the 
other Services are charged by law with furnishing certain support to the 
Army. These types of supports are listed in *C8 Publication 2 (UNAAF). 

For example, CONUS air defease is not the exclusive responsibility of 
either the Air Force or the Army- A CONUS air defense problem must 
consider Army surface -to -air missiles, Air Force manned interceptor*, 
and Air Force surface-to-air missiles. 

Current and protected capabilities of the other Services can be 
obtained from s number of different sources including the Five-Year Force 
Structure and Financial Plan maintained by each Service. The reviewer 
should bear in mind that functions such as air defense, the attack of sur- 
face targets, reconnaissance in the vicinity of the FEBA, and transportation 
within a theater are act the exclusive responsibilities of the Army. 
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11. ARE MDCT'JRES OF SYSTEMS (ORGANIZATIONS) CONSIDERED 

AMONG THE ALTERNATIVES? 

The reviewer should watch lor failure to consider appropriate alteiim- 
ti-es that are based on mixtures ot two or more v stem 3 (organizations) to 
combine the best features o f each. For example in comparing certain 
transportation systems one alternative for 8’ -face transportation might be 
a combinatlor oC truck, rail lad water systems rather than only a truck 
system. In the same manner, the study of a proposed new missile system 
might consider as an alternative a suitable combination of existing missile 
and gun systems and aerial fire support rather than only an existing missile 
system. 


12. ARE ANY FEASIBLE AND SIGNIFICANT ALTERNATIVES OMITTED ? 

A major contribution that a reviewer can make is to point out signlTl- 
cac*, and feasible alternatives that the study may have failed to consider. 

If any Of the answers to the previous questions on 11 Alternatives" are in 
the negative then it is possible that some feasible and significant titer na- 
tives were not considered. However, the reviewer must exercise judgment 
before criticizing a study for failure to consider all possible alternatives. 
There are practical limits on the time and manpower a vt - table for a given 
study. The relative importance o f the decision on the subject under study 
will also influence the number of alternatives examined. The reviewer 
should consider these aspects in determining whether feasible and signi- 
ficant alternatives have been omitted to the detriment of arriving \t sound 
recommendations . 

On the other hand , a large number of alternatives may only indicate 
that minor variations have beon considered as new alternatives. Excessive 
use of such minor variations as alternatives often beclouds significant 
choices. 
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DOCUMENTATION AND DATA 


13. IS THE STUDY ADEQUATELY DuCr T ENTED ? 

A key element of systematic analysis is sufficient documentation o I 
methods and sources so that with the same material, other study groups 
can arrive at substantially the same results. Without such documentation, 
a study appeals for acceptance solely on faith in the authority and expertise 
of the study group and without critical examination of the soirees and 
methods used to arrive at the recommendations. 

The test of adequacy can be applied by examining the models, data, 
assumptions, etc. , to determine if they are stated in 3uch a way that 
another study agency could trf.ee through the steps of the study and arrive 
at substantially the same results and conclusions. A study that is not, 
adequately documented will usually fare poorly when reviewed by agencies 
lacking the detailed knowledge of the problem that can sometimes compen- 
sate f..r poor documentation. Inadequately documented studies may 
require only slight additions to be properly documented. 


14 * AM THE FA C TS STA1 Q CORRECT ? 

It is usually neither possible nor necessary fo? a reviewer to verify 
all the factual material presented in a study, but it is advisable to spot 
check. Particular attention should be paid, where possible, to the factual 
material or. which conclusions and recommendations depend. If many 
error? are involved then a thorough verification of the facts presented may 
be in order . 

In reviewing factual material, its source should ha examined critically. 
For example, frequent use is nude of data contained in FM 101-10, 

" Organisation, Technical, and Logistical Data" and similar publications. 
The data contained in these manuals are usually averages of historical data 
obtained from certain kinds of operation? in specific theaters. The un- 
questioning use of these average figures »na> lead to erroneous conclusions 
because the use of an average hides significant variations that exist in the 
real world. A tank battalion does not aiways cover the same number of 
miles each day even over the same terrain. Further, the data contained 
in the reference manuals may not have been computed for the purpose re- 
quired in the study ’.id considerations important to the study may not be 
included ir. the calculations. For example, ammunition expenditure rates 
contained in FM 101-10 are based on World War II and Korean experience 
and organizations. The use o f these rates for projected operations in the 
1966-70 time frame would be questionable. 

Projection cf current operational experiences in*o future time frames 
should also be examined c'-itically. For example, a study useo an data 
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that an arm ad helicopter* a missions are A% escort, 3 % casualty production, 
and the remaining missions for suppressive fire. These data were obtained 
from experience in Viet Nam operations. This unquestioning projection of 
such data into future operations in other areas fails to allow for possible 
introduction of significantly new US and enemy tactics and may result in 
conclusions and recommendations on how better to cope with the last war. 


15. ARE THE FACTS STATED WITH PROPER QUALIFICATION? 

In addition to checking the validity of the factual material, it is good 
practice to check the factual material for completeness. Some material 
may be factually correct in isolation but may take on a different signifi- 
cance when other facts are added. Fpr example, it is true that infantry 
units can march at an average rate of 2. 5 miles per hour. However, this 
rate is valid only on relatively level roads. 


16. ARE FINDINGS AND DATA FROM FIELD EXERCISES AND FIELD 

TESTS USED? 

Field exercises and field tests can be excellent sources for effective- 
ness data. When used in a study, such data should be carefully examined. 
The reviewer should determine whether the data were obtained by measure- 
ments or by judgment of individuals and if similar data would likely be 
obtained if the field test or field exercise were conducted again by another 
agency or unit. The circumstances surrounding the field exercise or field 
test should be reviewed, where possible, to determine if any artificialities 
(there are always some in any peacetime operation) were of sufficient in- 
fluence to affect the results of the study based on field data. Field 
exercises usually have manv parameters and very few runs, therefore 
making it very difficult to single out cause and effect. 

Common artificialities that may significantly affect data from field 
exercises and field tests include: 

a. Inability to assess effectiveness of air 
defense fires ai:'-to-surface fires, and 
grcund-to-gTound fires. 

b. Lack of realistic levels of support 
from the other Services or other 
supporting units . Often this support 
is either not available or available 
in abnormally large amounts. 

c. Use of administrative breaks for rests, 
intensive resupply, and maintenance 
operations. 
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d. Unrealistic maneuver and deployment 
because of restricted maneuver areas. 

e. The units or quantities ci materiel 
tested are not a valid sample either 
because of inadequate size or of bias 
in composition. 

f. Poor or inadequate reporting of events 
of the exercise, 

g. Effect on actions of participants caused 
by use of only blank ammunition. 


17. ARE THE DATA FROM SUPPORTING WARGAMES VALID? 

Studies sometimes use the findings of wargames as facts. In eval- 
uating such fact,::, the reviewer should bear in mind the nature of wargames. 
Basically, a wargame involves a hypothetical situation in which two 
opposing sides interact in accordance with a set of more or less definite 
rules. In all forms of wargames, the play is determined either by mech- 
anistic rales or judgments made by individuals or both. These rules and 
judgments are based on assumed situations and known or assumed facts 
and system characteristics. Well planned and executed wargames are 
excellent teaching devices and provide the participants with good insights 
into the problem gamed. Such games, if well documented, usually provide 
a body of synthetic data which, when analyzed, provides clues to problem 
areas that need further investigation. 

In determining ti e validity of the findings of wargames, the reviewer 
should judge how well the game portrryed reality and should satisfy him- 
self on the validity of the judgments and assumptions used in the conduct 
of the game. The study should lay out for scrutiny the major judgments 
and assumptions used in the wargame. It is recognized that it is usually 
not possible to lay out all judgments and assumptions used in the wargame. 
In any case, the reviewer should weigh the dependence of the conclusions 
and recommendations on the findings of the wargame and consider whether 
other competent players playing the same game would have arrived at 
similar results. 


18. ARE THE PERFORMANCE CHARACTERISTICS VALID? 

Performance characteristics data are often the key elements in the 
determination of the effectiveness of a system. In evaluating the validity 
of performance characteristics, the source of the data should be examined. 
When the claimed performance characteristics are essential to ;he con- 
clusions and recommendations and the source of the data is not clearly 
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stated, additional information may be required from the agency that pre- 
pared the study. This may not be necessary if the study contains a 
sensitivity analysis of a reasonable range of values for the perfonmance 
characteristics. 

Performance characteristics baaed on a manufacturer' e claims are 
often too optimistic. Performance characteristics derived from tests at 
research installations also require examination. Sometimes, such per- 
formance characteristics are derived under controlled conditions thit 
neglect the man-machine relation that exists under field condition. Even 
performance characteristics derived from field tests should be examined. 
Such testa can, at times, produce misleading results due to artificialities 
caused by various peacetime restrictions such as safety regulations and 
choice of test areas. 

If faced with questionable performance characteristics that are key 
to the conclusions, the reviewer should consider: (1) performing a 
sensitivity analysis himself if his time and the data in the study permit; 
(2) requesting validation of the performance characteristics and 
sensitivity analysis. 


19. ABE ANY OF THE DATA DERIVED FROM QUESTIONNAIRES? 

The data obtained from questionnaires should be examined to determine 
the validity of the questions, the adequacy of the sample and statistical 
procedures, and the expertness of the personnel questioned. For example, 
one study cited data on the frequency of kinds of missions expected to be 
flown by Army aircraft in a conventional war. The data were based on a 
questionnaire completed by Army aviators at one Army post. There was 
r.o operational experience applicable to the study and an educated guess o* 
subjective judgment was in orde”. However, in this case, the judgment of 
those who order Army aviation n ssions flown (commanders, operations 
and intelligence officers) should have been elicited rather than the judgment 
only of those who execute the missions. 


20. ARE GUESSES AND INTUITIVE JUDGMENTS IDENTIFIED? 

At times it is necessary to fill in data gaps with educated guesses and 
intuitive judgment. These educated guesses and judgments should be 
identified in the study and not " swept under the rug. " The reviewer 
should evaluate these judgments and weigh their impact on the conclusions 
and recommendations. 
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COST 


21. IS THE COST MODEL IDENTIFIED? 

Every cost-effectiveness analyses contains a cost model. A cost 
model generates cost estimates by application of cost estimating relations 
and cost factors to specified physical resources. (For a further discus- 
sion on models in general see question 37.) This model can be very 
complex and computer assisted or it may consist of a few relatively simple 
equations readily computed by hand. The study should sufficiently identify 
the cost model so that the reviewer can determine how the total system 
cost estimates were derived from the material in the study. If the mater- 
ial in the study does not permit the reviewer to do this, then additional 
information is requited from the agency that prepared the study. 

The cost models are utilized to estimate the probable economic im- 
pact on the Service (or Nation) of introducing a new capability. For 
planning, these costs are normally stated in terms of research and de- 
velopment costs, investment costs, and operating costs. Research and 
development costs include those costs primarily associated with the 
development of a new capability to the point where it is ready for opera- 
tional use. Investment costs are those costs beyond the development phase 
to introduce a new capability into operational use. Operating costs are 
recurring costs required to operate and maintain the capability. 


22. ARE THE COST ESTIMATES RELEVANT ? 

Cost estimates depend on the problem ur-der studv and can rarely be 
obtained from books containing cost data although cost factors and cost 
estimating relations (CERs) can sometimes be found in such books. For 
example, a hypothetical study considers as an alternative a new kind of 
light infantry division which has been designed to the extent of an outline 
TOE. The answer to the seemingly simple question, " What is the cost of 
this new division?" depends on many factors including: 

Will it be an additional division to those already 
in the force structure? 

Will it replace an existing division? If so, what 
kind? 

Where will i‘ be stationed? e.g., in the CONUS, 

Pacific, Europe, etc. 

Will it have new Standard A equipment, or will 
existing pssets of Standard B type equipment be 
used? 
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Are there any existing Army units whose personnel, 
equipment, and fatalities can be used by the new 
division? 

The determination of which costs are relevant requires considerable 
analysis and judgment. It is not possible to prepare % universal list of 
costs that are always relevant. Ideally, a study should Indicate why cer- 
tain costs were considered relevant and others not. The questions that 
follow are designed to help the reviewer determine whether the cost 
estimates used in a study are relevant. 


23. ARE INCREMENTAL COSTS CONSIDERED ? 

Inherited assets are those resources such as installations, equipment, 
and trained personnel inherited from earlier systems which are phasing 
out of the force structure and are usable *n one or more of the alternatives 
under study. The costs which are usually pertinent for planning purposes 
are those costs yet to be incurred. For example, a study considers as an 
alternative the conversion of certain artillery* units from tube to missile 
weapons. In determining the incremental costs consideration should be 
given to the inherited assets of trained personnel, equipment, and facili- 
ties that are or can readily be made common to both units. 

Sunk costs are those costs already expended. These previously in- 
curred costs are normally excluded from costs presented in cost-effective- 
ness analysis. For example, a study considers as possible alternatives 
weapons systems A, B, and C, each with an associated research and 
development cost. Only alternative A is already under development. Tne 
cost already expended on Alternative A is a sunk ccst and the research and 
development cost of Alternative A in the study should be only what must 
vet be spent (to complete the research and development of Alternative A). 

An occasional error is the failure to consider the research, develop- 
ment and investment costs of existing systems as sunk costs. For example, 
in a hypothetic.''.! study of the conversion of certain artillery units from tube 
to missile weapons, one of the alternatives is retention of all of the tube 
weapons units. The cost of that alternative would not include the sunk costs 
represented by the research and de 'elopment and investment costs already 
expended in bringing those units into the force structure. 

Including the costs oi" inherited assets and ether sunk costs leads to 
distorted cost estimates with consequent effect on the conclusions and 
recommendations. 
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24. ARE DIRECTLY RELATED SUPPORT COSTS INCLUDED? 

Cost estimates of systems or organizations should include the pro- 
portionate cost of those other units or elements required in direct support. 
For example, the cost estimate of HAWK battalions should include the 
costs of the associated HAWK direct and general support detachments. In 
the same manner, the cost of aviation units should include a direct share 
of the cost of aviation maintenance units. Failure to include directly re- 
lated support costs may result in misleading cost estimates of alternative*. 


25. ARE COMBAT CONSUMPTION, REPLACEMENT/ CONSUMPTION, 

AND MAINTENANCE FLOAT COSTS INCLUDED? 

Cost estimates for the major equipment items should include not on] 
the operational equipment assigned to organizations, but also the costs 
for those additional items required for initial stockage as well as replace- 
ment items over the period in which the system is to be in use. (See 
question 32.) If the resource implications for procuring and maintaining 
authorized maintenance float, replacement/ consumption, and combat con- 
sumption stockage are excluded, the total costs of the system alternatives 
may be significantly misleading. (These levels of stockage are, of course, 
subject to logistics guidance.) For example, a common error is to in- 
clude only the cost of the basic load of ammunition and to neglect the cos', 
of the additional ammunition requirements for support of the weapon sys- 
tem or organization. The total ammunition required, to include peacetime 
training requirements and expenditures in the first part of a war until 
wartime production becomes available, must be purchased and stocked in 
peacetime , 


26. ARE ALL TRAINING COSTS INCLUDED? 

The resource implications <f training military personnel can be 
significant. Initial training costs represent the resources required for 
the training of personnel necessary for introduction of the alternative into 
the force structure. The availability of fully-trained personnel, as well 
as the number of personnel requiring complete or transitional training, 
are taken into consideration in determining the resources required. 

Annual ‘raining costs represent the resource implications for training re- 
placements. These replacements are required because of normal 
attrition. 

Training costs usually include such items as: (1) procurement of 
equipment utilized for training purposes; (2) construction of any neces- 
sary additional facilities; (3) operation and maintenance costs of the 
facilities; (4) the pay and allowances of the trainees. For example, the 
cost implications of communications-electronic equipment utilized for 
training ;■ utdoscs could be highly significant. 
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27. ARE CONSTRUCTION COSTS INCLUDED? 

The costs for additional installations or facilities are often overlooked 
yet these costs can be important. If the study does not include any con* 
struction costs and does not state how the facilities were obtained, then 
the reviewer must either satisfy himseli that no construction is required 
or take necessary steps to have the study corrected. 


28. ARK THE COST DATA REASONABLY ACCURATE? 

Although it is not usually possible for a reviewer to check all cost 
data for accuracy, le should spot-check and examine the sources of the 
data. 

Cost data furnished by manufacturers should be viewed critically. 
Experience has shewn tha* such data are usually understated, particularly 
for advanced systems. Advanced system costs stated as an exact figure 
rather than as estimated lower and upper values are particularly suspect. 

The basis of the cost data for advanced systems should be included in 
the study. There are a number of ways for arriving at such estimates. 
One commonly accepted method relates the cost data for the components 
of existing analogous systems to the cost of the advanced system. Unsup- 
ported cost data are suspect. 

Great accuracy in cost estimates is not required and often is not 
feasible. In fact, in dealing with costs of advanced systems it is usually 
more realistic to have a range of possible costs (upper ami lower values) 
rather thar the pseudo-accuracy of one cost figure which assumes no 
uncertainties in arriving at that figure. 


29. ARE COST ASPECTS OF ALL ALTERNATIVES TREATED IN A 

COMPARABLE MANNER? 

Inconsistency in handling the cost aspects of competing alternatives 
prevents an objective evaluation of their comparative or relative costs 
and usually leads to erroneous conclusions. It is not always possible to 
use the same cost estimating technique for calculating a cost element such 
as attrition replacements. This is often the case in studies involving 
alternative systems of other military services. For example, one service 
may calculate aircraft attrition replacement as a function of an activity 
rate (e.g. , per 100,000 flying hours) while another service may calculate 
it as a function of the activity inventory (3 percent of the active inventory 
per year). The reviewer should dete -mine that the final dollar estimate 
is related to the actual resource requirements for the alternative and that 
computational peculiarities do not distort the cost results. 
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Treating alternatives in a comparable maimer must not be carried to 
the point that costs which may be insignificant in one alternative are 
therefore not considered at all in other alternatives. For example, 
civilian personnel might not be used in one alternative but be required 
by another alternative in significant numbers. To exclude this cost could 
distort the results. 


30. ARE THE COST ESTIMATING RELATIONS VALID? 

Cost estimating relations may be crude factors, simple ex./apolaticn 
oi recent experience, or complex equations with many variables. In all 
cases, the purpose of a cost estimating relation is to translate a specifi- 
cation of a physical resource into a cost. The design of valid cost 
estimating relatiors is a complex subject beyond the scope of this publi- 
cation. However, several common errors made in establiening cost 
relations are discussed below. 

Cc3t estimating relations should be based on current data or distorted 
estimates may result. For example, the maintenance cost per flying hour 
for an Army helicopter has decreased significantly over the past several 
years as new helicopters have been introduced into the force structure. If 
the cost estimating relations used in a study were based on information for 
early Army helicopters (e. g. , 1948 through 1954 data) the maintenance 
cost per flying hour for a present system as well as for future systems 
alternatives could be distorted. 

At times a properly constructed cost estimating relation may be 
inapplicable. If the system alternatives are very advanced developments, 
the cost estimating relations based on the current technology may lead to 
false results. For example, the V/STOL aircraft concept represents a 
departure from aircraft currently in production. While many design 
characteristics may be similar to present aircraft, there may be a number 
of factors which could increase the complexity and hence, the cost of the 
aircraft; a cost estimating relation bated on ira present state-of-the-art 
may not be appropriate. 


31. IS AN AMORTIZED COST USED? 

Amortized costs reduce the total program cost of the system to an 
annual cost by taking the total operating cost of the program, adding to it 
the research and development and investment costs, and reducing the 
total to an annual basis bv dividing by the number of years of expected 
service life of the system. The same general procedure may be utilized 
to amortize the costs per month, per day, per sortie, etc. This approach 
disguises the differences between annual opera f lng costs resulting from 
shifting deployment patterns over the life of the system and from a varying 
set of inherited assets over time. This approach makes an arbitrary 


D-26 


AMCP 706-191 


allocation of “ e fixed coat* of a system over time. There is no basis for 
the assumption that the last year of system life must be charged with the 
same amount of RftD cost as the first year. The first year gets the newest 
technology, the last year, obsolete technology. Further, an amortized cost 
does not present a true picture of the total resource implications. If the 
system is to be in the force structure for say 10 years, the amortized 
annual costa may look relatively small, yet in reality there may be rela- 
tively large dollar coeta. It is the total cost of the alternatives which is of 
primary concern. 

The reviewer should attempt to convert amortized costs into total 
program coats and use such costs for comparative purposes. If this can- 
not be done readily from the material contained in the study, then additional 
Information is required from the study agency. 


32. WERE PEACETIME OR WARTIME COSTS INCLUDED? 

The results of n cost-effectiveness strdy may be very sensitive to the 
use made of peacetime and wartime costs. The use of peacetime costs 
only may indicate that System A is preferred while the same study, if war- 
time costs were used, may have concluded that System B is preferable. 

Peacetime costs may be defined as the costs associated with develop- 
ing, buying, and maintaining a capability for potential war during peacetime. 
Such costs also include combat consumption stocks (war reserves) to cover 
the period from the beginning of a war until wartime production is able to 
replace battle losses. Wartime costs are the costs of procurement after 
the war has begun, as is the cost of replacing the combat consumption 
stocks if the war terminated during the useful life of the system. 

In the case of general purpose forces there may be significant pro- 
duction of weapons and expenditure of resources alter a limited conflict 
begins (at in the Korean Conflict and the military assistance rendered to 
South Vietnam). In this case, wartime costs could be significant. How- 
ever, wartime costs are difficult to determine because of uncerU inty 
regarding the duration ot the war, loss rates, and missions undertaken. 

The reviewer should be guided in considering the pioper cos» approach 
(peacetime or wartime) by existing policy or directive from the agency 
directing that th«. study be made. 


33. WAS A WARTIME ORDNANCE COST PER MISSION USED? 

The use of a wartime ordnance cost per mission should be reviewed 
carefully. Variations on this approach include ordnance cost per target 
killed, per casualty and per sortie. Thu approach is usually deficient 
because of failure to consider al! the cost:, of putting into place ami 
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maintaining a capability for potential war throughout the projected life of 
the system in the active force structure. Often, this approach iucludes 
only the ammunition costa expended during a brief battle, and neglects the 
bulk of the significant costs associated with developing; bu>ing and opera- 
ting the system in peacetime. 


34. WAS AN AMORTIZED WARTIME -PEACETIME COST USED? 

In this approach the total peacetime cost of the system is reduced to 
an annual basis as explained in question al To this amortized peacetime 
cost is added the estimated annual wartime replacement/ consumption 
costs. No distinction is made between wartime and peacetime < ats. Th'« 
approach is deficient because: (1) it assumes the war will continue over 
the entire projected service life of the system; (2) the cost results use 
weighted wartime costs, (3) rtrtl v liuC and peacetime costs may not be 
commensurable; and (4) it does not present a true picture of the total re- 
source implications as discussed in question 31. 

Adding amortized costs in one stream to anotner annual cost stream 
infers that both cost streams represent the same total time duration. If 
this is not the case, then the two cost streams should not be added together 
because thev are incommensurables. Adding the amortized peacetime 
costs to the annual wartime cost implicitly assumes that the war will con- 
tinue over the entire " service life" of the system. If the peacetime costs 
had beer, amortized to a per day or per mission coct instead of a per year 
basis, the same result would hold, the inference being that the war would 
continue over the entire " service life" of the system. The implied assump- 
tion that tne war would >ast for the " service life" of the weapon system is 
questionable. 

C._sts computed by this method are weighted because wartime costs 
do no cover the same iengtn of time as peacetime costs. Such weighted 
results favor the shorter tin. • period— the wartime c^sts. It is only when 
the two cost streams are of equrl length that the costs results are not 
distorted . 

To assume, that wartime and peacetime costs are commensur .ble may 
be erroneous. This assumes a common measure between the values of 
le sou fees pr icured in wartime ami those procured in peacetime. During 
wartime, the cost of a r source may be quue different from that tn peace- 
time. Military b. .Iget constraints during peacetime ami physical resource 
constraints during wartime may produce entirely different sets of coifs for 
the same military resources. As a greater proportion of the national bud- 
ge! is shifted to military purposes during wartime, the scarcity of doHa^s 
lor military resources r.iav become relatively less or more than during 
peacetime. Comnu nsurabih v between wartime and peacetime costs will 
depend upon such urn ertamties as the tvpe and duration of war and whether 
, war will actual’v ix cur. 
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35. WAS A DOLLAR COST ASSIGNED TO THE LOSS OF HUMAN LIFE? 

Frequently, a study will assign a dollar cost to a human casualty. The 
loss of human life is certainly important in selection among alternatives. 
However, the value of a human life is incomnmnsuraole with the dollar 
costs associated with ~n alternative. It is belter to treat human losses as 
a separate measure without assigning dollar values. Manpower availability 
in both peace and war is very important but this problem cannot be properly 
treated only In terms of dollar costs. Men and dollars may not i>e inter- 
changed. 


36. IS THE SENSITIVITY OF CQ6T ASSUMPTIONS EXAMINED? 

In comparing costs of alternative systems, it is important to determine 
whether the results arc independent of the cost assumptions. For example, 
Wuuld ter. yea - , of peacetime operations as opposea to five make a signifi- 
cant difference in the relative cost* of the alternatives'* Would it make 
any difference if the procurement .eveis or number of units to be organized 
changed? Th? study should make clear the sensitivity of the cost estimates 
to the major cost assumptions. If the studv (ails to do this, the reviewer 
should attempt to determine if there . - any soon significant sensitivitv by 
rough calculation. 
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37, A RE THE MODELS ADEQUATELY IDENTIFIED AND EXPLAINED ? 

The conclusions and recommendations Oi a cest-effpctivenens 
analysis cannot be evaluated properly unless the models are adequately 
identified and explained, Every model portrays the real or expected 
world by abstraction and simplification in order to predict the outcome 
oi a possible action (see Glossary!. Therefore, the explanation of the 
model should be sufficient to provide ready understanding ot which as- 
pects of the real world are included in the model, which aspects have 
been omitted, and the underlying assumptions tor the abstraction. Basics 
ally , a good model emphasizes tne specific areas in which decisions 
are to be made by removing those relatively constant elements of the 
real or expected world that can be described with a great degree of 
certainty. 

The study should contain sufficient explanation to permit tracing 
die operation of the model from input to output. The detail should be 
sufficient to permit calculation of new results from different input values 
(sensitivity analysis). In cases wlici'e a model is machine-programmed, 
sufficient explanation should be provided for following the general logic 
of the program. 

38. ARE COST AND EFFECTIVENESS LINKED LOGICALLY? 

A properly structured cost-effectiveness analysis contains a number 
of models that link effectiveness and cost through logical interrelations. 
Usually there are some kinds of an effectiveness model, a system and 
organization model, a cost model, and a cost-effectiveness model. The 
exact nature and number of these models will vary with the problem. 

The study should provide sufficient information and explanation for the 
reviewer to follow the logic by which the models relate cost and effect- 
iveness. 

An effectiveness model relates measures of effectiveness to measures 
ot performance in an operational context. For example, ? study on com- 
bat vehicle weapons systems used pc a measure of effectiveness the 
probability of 1 , 2, 3, ... friendly tanks winning an engagement with 
1,2,3, ... enemy tanks under different tactical situations. This was 
related to performance measures such as muzzle velocity', warhead 
specifics, turret slev/rates, turret stability, hull characteristics, rate 
of fire, target acquisition accuracy, and others, under various tactical 
situations and rules for conduct of fire. 

A system ard organization model describes the physical resources 
required to pro - de the performance used in the effectiveness model. 

F..r example, in the combat vehicle weapons system study referred to, 
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this included th - physical description of each alternative, the complete 
vehicle, ammunition, armamen*, fire control, communications, TOE 
unit description, the support and maintenance requirements, and so 
forth, consistent with the planned operational concept, 

A cost model relate* dollar costs to the physical resources (and 
their peacetime activity rates) described in the system/organization 
model. The cost model apples cost estimating relations and factors. 

For example, the «ame study used the total future cost of aquisition and 
ownership (R&D, initial investment, annual operating) for yarious quan- 
tities of systems. Included in these total costs were not only the develop- 
ment and procurement of the preferred Item but also such additional 
costs caused by training of perjonnel, Deacetime ammunition use, equip- 
ment maintenance, etc. ($ee question 22). 

The cost-effect veness model finally relates the costs of each al- 
ternative to its effectiveness under varying assumptions. Depending on 
the criterion, the model may compare effectiveness and costs of alter- 
natives at equal cost, at equal effectiveness, or at different cost and 
different effectiveness (see page 9). The method and the techniques 
used to achieve this cost and effectiveness relation should be logical and 
explained. For example, in one anti-tank weapons study the equal 
effectiveness method (winning the duel - all pertinent factors considered) 
was employed. Effectiveness was related to cost by a numerical formula / 
for ca 1 culation of cost of achieving duel success at a given range under 
specified conditions. This permitted plotting the following graph: 



Figure 1. Equal Effectiveness Method 
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The graph r«h<>ws iht cost of winning a "duel", i. e. , killing the 
target at various ranges. (The graph portrayed above is highly simpli- 
fied. In the actual study rather than a simple line , a band was used to 
portray th j variance in costs for winning a duel at a given range. See 
Bibliography Item No. 1 , pages 13 - 1.7 for more complete description). 

39. DOES THE MODEL TREAT THE PROBLEM IN A SYSTEM 

CONTEXT? 

Most military systems have many subsystems, sub- subsystems and 
so forth. Models should provide for the proper relations among subsys- 
tems so that the full Implications of a change in one part of the system 
will be reflected in the rest of the system. For example, a model in a 
study of an airborne surveillance system must not only show tire inter- 
relations among the aircraft (or drones), the sensors and their main- 
tenance, but also the interrelations with the information processing 
functions to be performed on the ground. 

40. DOES THE MODEL ALLOW FOR ENEMY REACTION? 

It normally takes several years to implement fully a decision to 
deploy a new system. Therefore, the enemy should be considered to 
have time to adjust to our system decisions. A major aspect of the 
effectiveness of our system is the degree to which it makes such adapt- 
ation for the enemy either technologically difficult or economically 
unattractive. 

For example, a study of a proposed system was based on its incor- 
poration into tLe current force structure. The model for judging the 
effectiveness of the proposed system was dominated by current or 
recent conflict situations (e.g. , Vietnam, Korea, Europe). In using the 
model to evaluate the effectiveness of the future system only in the light 
of these current or recent conflict scenarios, the study failed to con- 
sider the steps that the enemy could take to counter the proposed system. 
(See question 6). 

41. ARE STRAIGHT EXTRAPOLATIONS USED WITHOUT PROOF? 

While straight extrapolations (linear relation) often do apply over 
limited ranges of performance, consumption, or similar planning figures 
based on averages of large numbers, they rarely apply to effectiveness 
or cost data. 

For example, the relation between the total weight of rations for 
one infantry division-month and the weight for 10 divi' ton-months is a 
straight extrapolation. The relation between the total cost of the first 
100 and that of the first 1,000 units of a new main battle tank is not 
linear or a straight extrapolation. If a missile system has 10 missiles, 
costs $1,000,000, and is 50% effective (on some valid measure), then 
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a missile system with 20 missiles, costing $2,000,000, will not be 100% 
effective but (at best) 75 To. 

42. ARE DETERMINISTIC AND PROBABILISTIC MODELS USED 

PROPERLY? 

A deterministic model (see Glossary) uses relations cf the type, 

"If A is 5, then B is always 8", A probabilistic model (see Glossary) 
uses relations of the type, "If A is 5, then B will be 6-10 in 50% of tin 
cases, 4 or 5 in 25% of the cases, and 11 or 12 in 25% of the cases". 

Cost-effectiveness analyses frequently require many intermediate 
calculations involving data. The indiscriminate use of specific values 
often creates what is in effect a deterministic model. In reality, the 
majority of the coefficients and planning factors used in models are 
only averages with variances and different degrees of confidence. The 
reviewer should try to identify the probable range of variance about the 
averages that are used as inputs and have at least an intuitive feeling 
about the confidence of the numerical results. 

Additionally, the reviewer should distinguish those cases in which 
a probabilistic model is needed to reflect the real world situation. 
Deterministic models are usually appropriate (1) when the planning fac- 
tor has an insignificant variance, such as weight of rations per day per 
man for large forces, (2) if the uncertain factor is multiplied by a point 
value, such as cost of $8,000 to $12,000 per man for a force of 20,000 
men, (3) a varying factor is multiplied by a linear function, such as an 
uncertain flying hour r?.te (<> g. , 2 to B hours per day) multiplied by a 
flying hour cost function of $20 a day plus $4C per flying hrnr, The deter- 
ministic technique is correct in these three cases because it will give 
the same most probable result as if probabilistic techniques had been 
applied.. Of course, there may still be a problem if the most probable 
result is not the only one of interest. 

Probabilistic models are used where the variables in the problem 
may assume, at any given time, any one value of a known range and 
frequency of values, as opposed to deterministic models which use 
fixed or average values all the time. There are two principal types of 
probabilistic models: static models using probability statements instead 
of other values, and dynamic (stochastic) models involving change. 

Some stochastic models use random numbers, representing change, 
to select values from frequency distributions for a given problem. For 
example, an analysis of a maintenance support organization may include 
a model which represents the demands for maintenance feffort placed on 
the support organization. Of any 100 jobs (demands), 20 will require 
1 man-hour, 30 will require 2 manhours, 10 will require 3 manhours, 

15 will require 4 manhours, 5 will require 5 manhours, 10 will require 
10 manhours, 5 will require 20 manhours, 2 will require 30 manhours, 
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2 will require 40 manhours, and 1 will require 80 manhours. This 
information is arranged into a cumulative distribution as shown below: 
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To represent requests for work, a two-digit random number, say, 

37, is drawn (from a table of random numbers or a random number 
generator)-, tne corresponding value is 2 manhours. The r°vt random 
number is, say 84, and the value is 10 manhours. This process con- 
tinues at some rate (which is probabilistic) and the requests for main- 
tenance are arranged (queued) in the order of simulated requests: 2 
manhours, 10 manhours, 4 manhours, and so forth. Available mainte- 
nance men would be assigned to requests under various rules, e.g. , 

1 man to jobs less than 4 hours, 2 men to jobs of 4 to 8 hours, etc. The 
model would keep track of the time elapsed between generating and 
completing a request for maintenance. In this manner, the relation of 
number of maintenance personnel and delay can be determined for various 
assumptions about demand for maintenance effort. 

The so-called Monte-Carlo model described above requires, however, 
a sufficient number of repetitions to obtain adequate information about 
the range of values of the solution. 

A static model using probability statements may, for example, 
apply in a study on aircraft vulnerability. The probability of survival 
for a specified time is given by the product resulting from he multipli- 
cation of the following probabilities: 

Probability of aircraft being detected 
Probability of aircraft being acquired by a weapon, 
if detected 

Probability of being hit, if acquired by a weapon 
Probability of kill , if hit. 

Probability data for each of the probabilities listed above are de- 
rived from tests and experiments. 

43. IS A ZERO-SUM GAME MODEL USED WHERE IT IS NOT 

APPLICABLE? 

A zero-sum two-per3on game is a conflict in which there are two 
sides and the gains of one side equal the losses of the other. Most con- 
fb'U situations do not justify the use of this type of model. For example, 
i,. a hypothetical study, the effectiveness of alternative US tank systems 
was based on a study’ of duels between US tanks and enemy tanks. Duel 


D-34 


WP 73C-1M 


result* were bused on the leases incurred by each side. An enemy loss 
of one tank was equated to a US gain of one tank The net US gain was 
used to determine th3 effectiveness for each alternative. 

Our gain is not the enenty' s loss. The situation is not always 
symmetrical. The attacker must move, the defender must Inhibit move- 
ment. Hence, the objective of a US tank may differ from the objective 
of the enemy tunk. In fact, other alternative concepts might inhibit 
enemy tank movement more effectively than would a UR tank similar to 
an enemy tenk. 

44. ARE THE MODELS INTUITIVELY ACCEPTABLE ? 

Models tend to become mathematical and many are difficult to under- 
stand even in their broad aspects. Yet, overly -simplified models tend 
to become superficial by limitation in choice of detail and omission of 
important variables. The objective of a good model is to be near enough 
to reality so that the model outputs can be used to predict some portions 
cf the future with an acceptable degree of confidence. 

Models can be tested by determining if they represent correctly 
known facts and situations not considered in the study. Conversely, if 
absurd facts and situations are introduced into the model, comparable 
absurd answers should be produced by the model. If the reviewer is 
aware of special cases in which there is some indication of the outcome, 
the model can be tested to determine if the results are in general agree- 
ment with the indicated outcome. Another test that can be applied, at 
times, is to vary some of the principal parameters and determine if the 
model produces results that are consistent and plausible. 
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EFFECTIVENESS 

45. ARE THE MEASURES OF EFFECTIVENESS IDENTIFIED? 

The study should clearly identify the standards or measures used 
for evaluating the effectiveness of the system or organization under study. 
If not explicitly stateo, the revie . .er should attempt to identify these 
measures from the material contained in the study. TTie conclusions 
and recommendations cannot be properly evaluated, particularly when 
the study is based on equal cost alternatives, without prior evaluation 
of the measures of effectiveness. 

46. IS THE EFFECTIV E NESS MEASURE APPROPRIATE TO THE 

FUNCTION OR MISSION? 

The reviewer should satisfy himself that the measures used to eval- 
uate effectiveness are appropriate to the function or mission of the 
system or organ! ration under study. Failure to use meaningful measures 
of effectiveness in a major contributing factor to unacceptable studies. 
Examination of the effectiveness measures requires analysis and sound 
military judgment. The example below illustrates one use of an effec- 
tiveness measure that was not appropriate. 

In a study of selected infantry and artillery weapons systems, the 
measure of effectiveness w»s a division firepower score. This score 

the firepower s cres cf the units within division. The 
firepower score of a unit was based on sustained rates of fire, effective 
width of burst, and the fragmentation area of the weapon in comparison 
with other weapons. Specifically, direct-fire weapons such as rifles 
were assessed in terms of probable hits per minute against personnel in 
the open. Mortars and artillery were assessed in terms of maximum 
effective range and lethal area coverage per minute. 

This use uf a firepower score was wrong for a number of reasons. 
Primarily, it failed to differentiate between the effectiveness of weapons 
when used for neutralization and when used to produce casualties. For 
neutralization. th« effectiveness Is ati-ngly dependent on burst rate of 
fire, incipient damage area produced by the burst, and ability to main- 
tain fire over the required time (the latter a function of weapon character- 
istics and ammunition requirements'. Or the other l.ind, casualty pro- 
duction depends strongly on tht probability of hit, wh en in turn depends 
on target acquisition and weapon guidance or accuracy. Thus, in this 
case, several measures must be used to ha* ■■ a valid analysis. 

The total di ision firepower 3Core useu in the study also assumed 
an inexhaustible and uniform supply of ammunition regardless of whether 
the weapon was a rifle company machine gun or a division general support 
artillery cannon. 
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47. DO THE EFFECTIVENESS MEASURES IGNORE SOMr. OBJECTIVES 

AND CONCENTRATE ATTENTION ON A SINGLE ONE? 

In the measurement of effectiveness, the reviewer should watch for 
any tendencies to concentrate on only one or two objectives. Sucn a 
situation indicates an unstated assumption that other objectives are 
unimportant. The resulting conclusions and recommendations, if imple- 
mented, may cause an imbalance and reduce the capability to achieve 
other objectives. 

For example, a study indicates that the moct vulnerable element in 
a line of communications system are the bridges in a rail network and 
measures effectiveness of deployment of given air defense units by 
degree of protection afforded railway bridges. In evaluating tao overall 
effectiveness of the air defense deployment, the s'udy failes to consider 
that the vulnerability of other elements in the 'lift ; rf communications 
system may be greatly increased by the redeployment of the air defense. 

A possible test for effectiveness measures suspected of concentrating 
on a single objective is to evaluate them against a hypothetical obviously 
absurd weapon or device that does only one job Valid measures of 
effectiveness should show an absurd hypothetical weapon or device In its 
true light. 

48. ARE PERFORMANCE MEASURES MISTAKEN FOR EFFECTIVENESS 

MEASURES? 

Measures of performance characteristics are sometimes miscon- 
strued as measures of the ability of the system or organization to ac- 
complish its function. Performance characteristics may contribute 
one of the many inputs required to achieve the effectiveness of the system 
or organization as a whole. For example, ± n speed of movement or 
mobility of a unit is only one aspect of the unit’ s capaoility to accomplish 
its function. The speed at which a unit can attack the enemy is not In it- 
self a measure of the ability of the unit to defeat the enemy. The weapon 
with the smaller CEP is not necessarily the more effective weapon; the 
relation of lethal radius to CEP may be more significant. Other factors 
that must be also considered in weapon effectiveness include target ac- 
quisition, weapon guidance, and target size. 

*». IS THE EFFECTIVE. JESS CALCULATED ON THE BASIS OF EITHER 

A COOPERATIVE ENEMY OR AN OMNIPOTENT ENEMY? 

Neither basis is valid, fhe enemy should tv- expected to adjust 
his decisions to our own planning as much as his resources permit. An 
unstated assumption thzt the enemy is inflexible in the face of our changes 
is a common error in cost-effectiveness studies. 

For example, a counter-guerrilla study used a scenario in which the 
hostile guerrilla forces retreat to a mountain redoulp to be surrounded 
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by US troops air-lanaed by helicopters. This scenario makes convention- 
al Unties palatable in counter-guerrilla warfare, but is hardly realistic. 

A capable guerrilla leader should not be expected to use such disastrous 
tar tics. Adapt ion of enemy tactics (e.g. , rapid dibp’raal) in face of 
the new U3 capability for air landing is certainly feasible. A comparable 
adaptation to the enemy capabilities was illustrated during World War Ii. 
German air defense analyses prior to tha; war were based on the attack- 
ing aircraft using certain altitudes that were optimum for the air defense 
batteries. Allied bomber aircraft did not oblige end avoiued the "optimum" 
altitude range. 

Some studies assume maximum future enemy capability in all weapon 
areas. The enemy <- , »anot simultaneously maxi nize all of this capabili- 
ties if constraints physical resources and budgets are present, parti- 
cularly in the case peacetime budgets. If he maximizes his strategic 
force?, he will have to limit hie tactical capabilities, and vice versa. 
Alternatives, where appropriate, should be pitted against a variety of 
enemy postures and the choice should make none of these postures parti- 
cularly attractive to the enemy. 

In theory, the enemy can counter every system we design and our 
effectiveness will not be sufficiently high to warrant a positive decision. 

The real question is: how much does it coat the enemy in time and 
resources to effect a direct counter? li the price is very high he will 
probably seek other lesser alternatives. {See question 6). 

50. IS THE EFFECTIVENESS MEASURED BY ANALYSIS OF WARGAMES? 

When effectiveness is measured by analysis of wargames the review- 
er should look to sensitivity analysis of the results As a rule, wargamea 
are a questionable means for measuring effectivenet s because of the 
difficulty of testing the sensitivity of the results. To do so means 
challenging the effect of changes in players, referees, .'cmmunications, 
as well as o*Wf . ( 3 ^ question 18. i 

51. IS THE EVALUATION OF EFFECTIVENESS BASED ON STRAIGHT 

EXTRAPOLATION? 

Occasionally a study may evaluate effectiveness by straight (linear) 
extrapolation from the measureme.it of effectiveness of a small unit. 

For example, a hypothetical study may show that 6 riflemen can destroy 
10 targets. An extrapolation that states 100 targets can be destroyed by 
60 riflemen is not justified without sup|X>rUng evidence, ine variables 
in target and fire distribution are not necessarily th'- same in both 
cases. Further, in a force of 60 riflemen the percentage who will 
actually (l. _ at targets may not be the same as for a force of 6 riflemen. 

Another error in straight or linear extrapolation is disregard of the 
element of diminishing returns or marginal utility. For example. 200 
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mi s si es do not tignify twict as mvch ♦.-iiectivene. & ?'■ GO nTssiles if 
there are only 50 targets. i arji'Tir.un all targets not of equal 
value or importance. 

52. ARE THE OPERATIONS OF GTHE.V SERVICES IGNORED' 

In measuring thv effectiveness cl a system or organization, consid- 
eration must be given to the operations of othc. Services, where appro- 
priate. Failure to do so i« the equivalent of making the erroneous un- 
stated assumption Fiat ornj the Army will participate in the operation. 
For example, the measurement of effectiveness of Army air aefense 
operations mur; consider the communications, command ard control, 
and IFF aspects of operations w'th the U.S. Air F rce and t’lied air 
forces. Further, the effectiveness of certain Army operat ons is ae- 
p*m?ent upon the degree of air superiority ac. ieved by the Air Force. 

The ability to aohitv* this air sutler for it. and the degree of dependence 
upon it should be examined. (See question 10. ) 

53. IS THE IMPACT ON OTHER ARMY OPERATIONS IGNORED? 

In measuring the effectiveness of a system or organization , the 
ofiect*. on other Army operations should be considered. For temple, 
the use of tactical nuclear weapons i, a certain manner may at complies 
its function by stopping enemy ground movem-nt. However, the judg- 
ment of the effect! venees of the system should also examine the effect 
on ti-e ground movement of U. S. units. In the same manner certain 
protective clothing may be effective against enemy chemical agents. 
However, the clothing may cause such body heating that it can only tx* 
worn for very short periods. 

54. ARE SCME ASPECTS OF EFFECTIVENESS INCOMMENSURABLE 

OR UNMEASURABLE ? 

The reviewer should examine carefully the treatment of incommen- 
nurables and unmeasurable aspects of performance in the total measure- 
ment ui effectiveness. Misleading measures of effectiveness are now 
often obtained by quantifying such aspects as morale, or leadership. At 
times, th? only practicable solution may be a Qualitative tu* ussion of 
these factors. 

55. DOES THE EFFECTIVENESS OF A FUTURE SYSTEM T.nXE INTO 

ACCOUNT THE TIME DIMENSION? 

The effectiveness of proposed future systems is often dcoeadeat 
upon when they can be available fur operational use and the total opera- 
tional life span of the systems In examining the effect of the time 
dimension upon effectiveness, particular attention should be given to 
(1) the time betwot :. the present and tnc initial operational availability 
of the complete s> stem, and *2’. 'he latter .*srt of the system ^-rational 
life span. 
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For example, the effectiveness of Weapon Y, deployable In 1972, 
is compared with that of the current Weapon X. Weapon Y is judged to 
be more effective and requires entirely new support equipment not 
compatible with that of Weapon X. This equipment cannot be operation- 
ally available until 1974, It is very possible that the changeover from 
X to Y implies a d>p in effectiveness during the 1970-74 interval. The 
old weapon is becoming obsolete ar.d the new one is not fully effective. 

A quick fix means may he needed to bridge this gap and must be charged 
to the cost of X and Y. 

In another case involving the time dimension, Weapon B, deployable 
in 1972, replaces Weapon A and is designed to perform the same mission 
more effectively. It is stated to have an operational life of 15 years. 
Effectiveness is calculated on the basis of the 1972 environment. In the 
1972 to 1987 period (the operational life of B) the international environ- 
ment, nad hence the missions may undergo major changes. In fact, the 
mission for which A is designed may already be on the decline. Effective- 
ness is not always constant but often must be related to time. 

It is necessary to recognize mat missions do not remain fixed. 
Effectiveness should not be evaluated on the basis of either a specific 
probability of the continuity of the mission or of a specified new mission. 
Rather, the system should be judged on its ability to adjust to such 
changes. 

Similar comments apply with respect to changes in technology. 
Breakthroughs cannot be predicted very successfully. Nevertheless, 
certain trends are noticeable. For example, anti-tank weapons have 
improved more rapidly than tanks since World War n. The sensitivity 
of the system to jumps in technology is a vital input to effectiveness 
evaluation of massive long lifetime systems. 

56. 'IRE EXPECTED AND AVERAGE VALUES USED INCORRECTLY 

TO MEASURE EFFECTIVENESS? 

It is an error to employ an expected alue or average as part of 
a measure of effectiveness if the objective really requires a specif ied 
minimum. In such a case, the posable variances, or dispersions about 
the average, consitute an unacceptable risk for any single event. 

This risk is unacceptable even though over many events the results will 
average out to the expected value. 

For example, assume that at the same cost, air defense System A 
destroys trom 0 to 99 of 100 approaching enemy aircraft bu f on the 
average de£ ;roys 50. System B, on the other hand, destroys from 25 to 
35 of 100 approaching aircraft with an average destruction of 30. The 
risk associated with the possibility that, in any given individual attack 
by 100 aircraft, System A may not destroy any aircraft at all, whereas 
System B can be counted on L destroy at least 25 aircraft, makes A an 
unacceptable system, if the objective is air defense. If the objective 


D-40 


AMCP 705-191 


were destruction of as many enemy aircraft as possible over some 
period of time but without regard to their damage to us, (an unlikely 
objective) System A would be preferred. 

57. IF QUANTITATIVE MEASURES 01 EFFECTIVENESS ARE UN- 
ATTAINABLE, is A qualitativecompArison FEASIBLE?" 

There are times when the effectiveness of a system or organization 
cannot be presented adequately in quantitative terms. This situation is 
common in comparison of general purpose forces such as in studies of 
alternative divisions, A study that assigns numerical values to effec- 
tiveness of general purpose forces should be examined carefully. 

One study eom£ ed alternative divisions in terms of numerical 
scores. Each o' the *..x basic factors (firepower, intelligence, mobility, 
command/coutrol/comir mic&tions, logistics, survivability) was given 
a numerical value and the&e values were summed for each alternative. 

The resulting sums were compared as effectiveness measures. These 
numeric^ values are likely to be meaningless because die six basic 
factors are inputs and not objectives. They combine in undetermined 
ways to make up the effectiveness of tactics. The tactics, in turn, com- 
bine to evolve strategies. For example, deception tactics strongly in- 
volve the basic building blocks of intelligence, command/cont.rol/commu- 
nicati'ws . and mobility. However, this does not mem that wo can simply 
add up so-called scores of these three basic factors and thereupon com- 
pare the deception capability of various alternatives. 

A qualitative comparison is possible, however. Various pertinent 
aspects can be described and characterized by ''yes-no" or "good-fair- 
poor. " A tabular comparison can be useful in weeding out some alter- 
natives. It may be justifiable to say that Alternative A is more effective 
than B (denoted ^) in a certain characteristic, even if it is not known 
whether A is times or twice as effective at’ B. If it can be determined 
that ^ and g we have a partial ordering i. e. , we cannot distin- 

guish between B and C but either io inferior to’ A, we may obtain a 
grouping as follows: 


A 

E 

Let us reconsider the example of the deception tactic. Its key in- 
gredients are mobility, command /control /communications, and intelli- 
gence. Suppose we know that Division A is more mobile than B, there- 
fore, jj-. If we shouxo arrive at the same ranking for the other two 
basic meters, then we conclude thai 4 is also true for live deception tactic. 
On the other hand, it may be that 4- t° r mobility and ^ for intelligence. 
Then no statement can be made for the relative ranking of A and B for the 
deception tactic. 
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In this manner, tactics of interoot can bo investigated and valid 
partial orderings of alternatives obtained. We may find dominant alter- 
natives. Suppose we obtain: 

Command, Control, 

Mobility Intelligence Communications 

A, P E,A E 

C B C, D a7b7c 

P,E D 

We have now learned that D is dominated by A for all three basic factors, 
and hence for the deception function. So D can be eliminated if all al- 
te rnatives have equal cost. It should not be assumed that rankings of 
alternatives with respect to the tactical level can only be derived by 
buildup and integration from the basic level. There may be direct quali- 
tative comparisons with respect to, ray, deception effectiveness as a 
result of wargames or field exercises. A combination of both buildup 
and direct approaches would probably prove most fruitful. 

The reviewer should recogrize that :/hile cost-effectiveness analysis 
is performed preferably by quantitative analysis, there are limits to 
suboptimizing or idealizing the problem to make it am enable to quantita- 
tive analysis. When carrie'i too far, the quantitative results are often 
only of academic interest and offer little or no help to the decision maker. 

58. IS THE EFFECTIVENESS SENSITIVE TO CHANGES IN ASSUMPTIONS? 


The effectiveness derived in most sir dies is usually dependent to a 
degree on the assumptions. The reviewer should isolate the degree of 
dependence and determine If it is acceptable. Generally, 
a good study will isolate this dependence, where it exists, and lay out 
tile degree of dependence by various kinds of sensitivity or contingency 
analysis. The assumptions dial most commonly influence effectiveness 
and are often not subjected to contingency analysis concern the locale, 
the time and level of warfare, and enemy forces and tactics. 

a slight change in any of these assumptions may produce significant 
changes in the effectiveness measured. For example, additions of a new 
ECM band width to the enemy’s capability may drastically degrade an 
otherwise cutstanding U. d. system. (See questions 5 and 6. ) 
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CRITERIA 


59. ARE THE CRITERIA IDENTIFIED? 

The criteria, or tests of preferredness, are the basis for the con- 
clusions and recommendations. The criteria should be stated specifi- 
cally and clearly. U this is not the case, the reviewer should attempt 
to Identify the criteria from the material contained in the study. When 
this does not prove possible, consideration should be given to having the 
study returned for futher clarification. This is particularly important 
if the study is also to be reviewed by agencies outside the Army. 

60. ARE THE CRITERIA CONSISTENT WITH HIGHER ECHELON 

OBJECTI VES? 

No matter what the concern of a study, the subject falls into a 
larger framework. For example, problems of air defense of the CONUS 
are aspects of the larger problem of restricting possible damage to the 
CONUS to certain levels. The design of artillery systems is part of the 
larger problem of design of land battle forces. Therefore, the reviewer 
must determine if the criteria used in a study are consistent with higher 
level objectives. This requires good military judgment and the necessity 
to examine the larger context of die problem. If the study criteria are 
not consistent with the objectives at the higher level then the wrong 
problem may be solved. Overall Army objectives are contained in docu- 
ments such as the Basic Army Strategic Estimate (BASE), Army Force 
Development Plan (AFDP), Army Strategic Plan (ASP), and toe Combat 
Developments Objective Guide (CDOG). 

An example of incorrectly chosen criteria is illustrated in the use 
of mobility as the sole criterion in the selection among different organi- 
zations. A study could conceivably demonstrate that organization A can 
be made more mobile than organizations B and C with a lesser expendi- 
ture of resources. Yet A may not be the preferred organization because 
the mobilify was achieved by degrading other factors that contribute to 
the higher objective of efficient control of conflict situations (e. g. , fire- 
power, sustainability, etc,). 

61. ARE THE CRITERIA TOO GENERAL ? 

Generalized criteria are suspect. For example, a study may state 
that the criterion is "the system with maximum military worth* or the 
"best system". Such generalizations are meaningless and cannot be 
related to the analysis as can a good criterion such as "the minimum 
cost of maintaining a [ specified j level of transport capability over a 
[ specified] time span." 
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62. ABE THE CRITERIA OVERDETERMINED? 

Overdetermined criteria lead to erroneous conclusions. A criterion 
that states "to maximize the damage to the enemy while at the same time 
minimizing the cost to the U. S." or "causing the maximum amount of 
casualties with the least expenditure of ammuniticn" suggests that some- 
thing can be obtained for nothing. It is impossible to maximize gain and 
simultaneously minimize cost. It is not possible to increase effectiveness 
without some increase in resources (cost). The minimum cost is to do 
nothing— and achieve no effectiveness. Occasionally it may turn out that 
system A is both more effective than system B and costs less. However, 
system A will not be both more effective and cost less when compared 
with additional alternatives. The dagger of using an overdetermined 
criterion, such as the one described, is that it leads to invalid compro- 
mise criteria by using some erroneous constraint on effectiveness or 
cost in an effort to make an impossible test seem feasible. 

63. ARE GOOD CRITERIA APPLIED TO THE WRONG PROBLEMS? 

At times a valid criterion for cne element of the problem is incor- 
rectly applied to the total problem. For example, a hypothetical study 
involving proposed surveillance aircraft shows that aircraft A offers 
greater mission flexibility than aircraft B at the same cost and is there- 
fore preferred. In this case, the choice of aircraft is not the real 
problem. The subsystems carried by the aircraft are really more 
crucial. The all-weather sensor effectiveness and avloJcs cost may 
even determine whether there should even be an aircrcit A or B. 

64. IS THE ABSOLUTE SIZE OF GAIN OR COST INGNORED? 

If the absolute size of the cost of a system alternative, or the 
effectiveness to be achieved by it, is not given or is incorrect, the 
analysis often leads to wrong conclusions and recommendations. For 
example, cost-effectiveness curves for two hypothetical system alter- 
natives are given below: 



ENEMY TARGETS DESTROYED 
(EFFECTIVENESS) 


Figure 2. Cost-Effectiveness Curves 
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In tills situation, at low levels of effectiveness, alternative A is 
preferred (up to about 70 enemy targets destroyed); at larger levels 
of effectiveness alternative B is preferred (from about 70 to 110). If 
tiie capability to destoiy more than 110 enemy targets is to be achieved, 
then neither alternatives A or B is preferred or even acceptable. The 
crucial question is how many enemy targets are required to be deotroyed. 
If the number of enemy targets to be destroyed or cost limits are not 
indicated, there is no real basis to recommend either rl tentative A or 
B, or some other alternative. 

Either the study should be based on an absolute size of gain or cost 
required or the study should present a cost-effectiveness curve (or points) 
from which decisions can be made. If the study presents a cost-effective- 
ness curve as shown above, the envelope (indicated by line of X* • U the 
graph) along the bottom says, "This is a curve which gives the most for 
tiie resources expended, and other things have to be taken into consider- 
ation at higher levels to determine what the absolute gain (number of 
targets destroyed) should be or the maximum resources (cost) that can be 
made available. " 

At times, studies ignore absolute size of gain or cost and use effec- 
tiveness- to-cost ratios. The flaw in tiie use of such ratios is the absence 
ct ary specified level of effectiveness required or resources available 
as cuscussed above. If the level of activity is fixed, a ratio may be use- 
ful in ranking among alternative systems. However, tiie effectiveness- 
to-cost ratio criterion is often applied when tiie level of activity is not 
fixed. For example, in tiie graph above alternative A destroys in enemy 
targets for $1 million, and alternative B destroys 100 enemy targets for 
$25 million. If tally this information were converted to effectiveness-to- 
cost ratios, alternative A would have a ratio of 10:1 and alternative B, 

4:1. Which is tiie preferred? If one did not look at the absolute level 
of effectiveness required to achieve the military task but only at the 
effectiveness-to-cost ratio, then alternative A would be preferred. The 
selection of alternative A on tills basis may be correct, but only by 
coincidence and is obviously wrong when die system must be capable of 
destroying more than 70 targets. 

Until the absolute level (magnitude) of effectiveness or the absolute 
level (magnitude) of the cost is specified the preferred alternative cannot 
be determined. The effectiveness-to-cost ratio can be misleading and, 
at times, a dangerous criterion. 
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CONCLUSIONS AND RECOMMENDATIONS 

( 

65. ARE THE CONCLUSIONS AND RECOMMENDATIONS LOGICALLY 
DERIVED FROM THE MATERIAL CONTAINED IN THE STUDY? 

The conclusions and recommendations should be derived logically 
from the material contained in the study. Some studies, unfortunately, 
draw conclusions based on previou* studies and materials that are not 
filly docMnj**"*ed w<th<n the study (mention ir a bibliography is hardly suf 
flcient). If input from another study is essential, it should be documented 
and explained in detail. This requires, at least, a statement as to validity, 
scope of application and uncertainty which is associated with the particular 
input. 


The determination of whether the conclusions and recommendations 
follow logically from the material in the study is a matter of Judgment by 
the reviewer. In making this judgment, the reviewer should consider 
whether other prudent ttudy agencies would probably arrive at substan- 
tially the same conclusions and recommendations given only the material 
contained in the study 


86. HAVE ALL THE SIGNIFICANT RAMIFICATIONS BEEN CONSIDERED 
IN ARRIVING AT THE CONCLUSIONS AND RECOMMENDATIONS 
C ONSIDERED ? 

Sometimes a study fails to consider all the pertinent ramifications in 
arriving at the conclusions and recommendations. These unconsidered 
ramifications may either influence the validity of the conclusions and 
recommendations of the study or tac decisions to be made as a result of 
the study. Th«'*se ramifications are often referred to as " spillovers. " 

For example, a hypothetical study recommended adoption of an engire 
requiring a new type of fuel, the Army supply system to include supply , 
storage and transportation operations would be affected. Spillover effects 
are not always negative. For example, the adoptior of dehydrated rations 
to achieve greater shelf -life may also reduce construction and transporta- 
tion costs because of the smaller unit volume of dehydrated food. 

Other ramifications that arc sometimes neglected are factors that 
ahould be considered jointly with the problem under study. At timer, con- 
sideration of such joint decisions could effect the conclusions and 
recommendations of the study. For example, a study may recommend 
adoption of a new weapon system to fulfill a certain function. However, the 
study may negioct to examine the maintenance support and the maintenance 
units that would have to be in existence concurrently with the proposed 
weapon system. The resources required to organize and maintain the 
mait&enarce system will influence decisions on the proprsed weapon system. 
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If significant ramifications are uncovered that are cot adequately con- 
sidered, the reviewer should, If possible, determine the effects of these 
ramifications on the conclusions and recommendations. (See question 2. } 


67. ARE THE CONCLUSIONS AND RECO.'ttTENDATIONS REALLY 

FEASIBLE IN THE LIGHT OF LOLITICAL, CULTURAL, POLICY OR 
OTHER CONSIDERATIONS? 

In reviewing the conclusions and recommendations of a study, it is 
necessary to be cognizant of the real world in which the Army must operate. 
At times some recommendations of a s+vdy may apoear to be eminently 
feasible from a strictly economic or military view, but really are not so 
in the light of other considerations that Influence military operations. For 
example, a particular toxic chemical munitions system may be demon- 
strated to be superior, considering cost and effectiveness, to a high 
explosive munitions system for accomplishing a certain function. However, 
because of national policies on employment of toxic chemicals, the adoption 
of the high explosive munitions system may be the only feasible solution. 

The reviewer should also consider the impact of policies that may not 
have been known to the agency that prepared the study or were promul- 
gated too late to influence the study. 


68. ARE THE CONCLUSIONS AND RECOMMENDATIONS RELATED TO 

THE LIMITATIONS OF THE STUDY? 

In evaluating conclusions and recommendations , the reviewer should 
bear in mind the limitations of the study. Studie' is a rule, have 
varying degrees of limitations. The more cor uo. .pcs of limitations 
include inadequate data base, criticality of assumptions, criticality of 
uncertainties and validity of cost and effectiveness models. While the 
limitations may be treated within the study, the dependence of the con- 
clusions and recommendations on the limitations is sometimes neglected. 
For example, the study conclusions and recommendations may depend 
upon the validity of particular assumptions but this relation may not be 
pointed out. 

It may be advisable for the reviewer to refresh his memory on the 
?*udv limitations, particularly when the study is voluminous, before 
evaluating the conclusions and recommendations. 

69. DO THE CONCLUSIONS AND RECOMMENDATIONS INDICATE WT \8? 

Studies sometimes unwittingly reflect bias because of parcel 
institutional interests. To assist in detecting bias, tbs reviewer . v<ulH 
consider the relation of the agency that prepared the study end tne e e u 
of the implementation of the study recommendations. If such implex . 
tion does not appear to further what are generally considered to be the 
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particular Interests of the preparing agency, then one occasional form of 
bias is probably not presant. Another test for bias is to judge whether 
substantially the same conclusions and recommendations would be reached, 
based on the material in the study, by another study agency. Bias is 
often displayed by arbitrarily excluding certain reasonable alternatives, 
maximizing selected enemy capabilities, treating significant uncertainties 
as assumptions, and in selection of effectiveness criteria. 

A relatively minor form of bias is sometimes found in the use of 
prejudicial adjectives. Unnecessarily referring to all Air Force fixod 
wing aircraft as " long take off and landing" aircraft is an example. This 
type of bias may be prejudicial to the interests of the Army when the study 
is reviewed by non-Army agencies. 


70. ARE THF CONCLUSIONS AND RECOMMTIwATiONS BASED ON 

EXTERNAL CONSIDERATIONS? 

At times, recommended selections among alternatives must be made 
in the face of great uncertainty. A study may find that several alterna- 
tives exhibit similar cost -effectiveness, but the results are very sensitive 
to the values assigned to the inputs. In this situation some studies arrive 
at conclusions and recommendations based on considerations other than 
those studies. In other words, the study agency is stating that after 
having made the analysis, the application of the criteria does not lead to 
preference, but indifference, among the alternatives and therefore the 
issue was decided on the basis of other unstudied criteria. In situations 
of this Kind, when recommendation of an alternative is necessary, sensi- 
tivity 10 new criteria must be fully studied. 


71 . ARE THE CONCLUSIONS AND RECOMMENDATIONS BASED ON 

INSIGNIFICANT DIFFERENCES? 

At Hmes a study will presort one alternative as having the highest 
value oi effectiveness of the measures applied to all alternatives. The 
difference in effectiveness among the " optimum" alternative and the 
other alternatives should be examined. U the differences are relatively 
slight and probabiy no greater than the uncertainties in the data, then 
other grounds should also be demonstrated for selecting among the alterna- 
tives that are close in effectiveness. 


72. IF PRIORITIES ARE LISTED. ARE THEY STATED MEANINGFULLY? 

Conclusions and recommendations often list items of equipment in 
order of priority of recommended procurement or adoption. The use of 
this technique without explanation, particularly for mate' ic!. is often 
poor because it provides no basis for s decision. For example, s study 
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may conclude that In order to accompliah certain ftinctions, infantry unite 
should be equipped with specified items of equipment that are listed in 
order of priority. Assume that the items found necessary by the study for 
infantry units to accomplish the required functions are, in order of 
priority: 


(1) Seven League Boots 

(2) Disintegrator Ray Pistol 

(3) Universal Viewing Device 

(4) Camouflage Suit (makes ths wearer invisible) 

Although the study concluded that all of these items are required, the 
listing of pi irities without sty quantitative considerations oouii have any 
of these meanings: 

s. Day all of the Seven League Boot* required. Then, aa resources 
are available, buy all of the Disintegrator Ray Pistols reoulred. Continue 
down the list of priorities in this manner until the available resources are 
exhausted. This meaning also infers that even though all « items are re- 
quired, the Army can do without the lower priority items If sufficient 
resources are not available to procure them all. For example, with 
limited resources it Is better to have all Sewn League Boots and none of 
the other items rather than some of each item. 

b. Buy all 4 items at once but spend more money an Sevan League 
Boots than on Disintegrator Ray Pistols and even less amounts on Universal 
Viewing Devices and Camouflage 8utts. 

When faced with this kind of situation, corieidemtion should be given 
to returning the study -to the preparing agency for further recommendations 
on how much should bo allocated to each item for various budget levels, 
either given or assumed. 


73. ARE THE CONCLUSIONS AND RECOMMENDATIONS INTUITIVELY 

SATISFYING ? 

When the conclusions and recommendations of tbs study are not 
intuitively satisfying, the reviewer should attempt to isolate the cause.If 
the study fails to demonstrate by data, models and other means that the 
reviewer's intuition was wrong, then further examination is required to 
determine if some subtle considerations have not been considered b eca u se 
of oversimplification or other reasons which the reviewer intuitively knows 
are pertinent. 
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INTRODUCTION 


Nisterieal Btckgrouni 

This report summarizes the continuation of the work in analysis of com- 
munication 'system effectiveness reported in NEL report 1323 1 

Many reports such as the foregoing are available to guide analysts in 
evaluating and analyzing various system configurations. Some of the reported 
methods call for many judgment decisions on the part of the analyst, and in 
some cases require cost data that are impossible to obtain. Also, some methods 
permit grouping all the system characteristics into a single "figure of merit," 
or measure of effectiveness. The latter method is often misleading and meaning- 
less to the system analyst and decision maker. Other methods allow system 
effectiveness to be specified in terms of multiple measures of effectiveness. 
Several reasons for using multiple measures of effectiveness are contained in 
reference 1, as well as in appendix A. 

Single methods are not used in this report, as they are considered un- 
realistic in the evaluation of complex systems that have multiple objectives. 

Another method for analysis of cost effectiveness is one that realistically 
represents the pertinent sy stem characteristics and costs, is capable of being 
implemented, and can be used in a meaningful manner by the system analyst and 
decision maker. (Unfortunately, the more realistic and representative a model 
is, the more difficult it is to implement, and often compromises have to be made 
because of the time required for analysis.) It is this method that is being 
developed. 


Statement of Problem 

The ultimate objective of the work in communication effectiveness is 
twofold - - to develop a communication-system resource-effectiveness method, 
and to perform resource-effectiveness analysis of communication systems. 

The method is intended for use as a management tool and as a design 
tool. As a management tool, the method can be used in the preparation of 
Proposed Technical Approaches (PTA), Technical Development Flans (TDP), 
and Detailed Action Plans (DAP) to: 

1 Determine resource effectiveness of a set of technical approaches, 

2. Establish performance estimates, and 

3. Conduct resource-effectiveness trade-off studies. 

As a design tool, the method can be used to: 

1. Specify system characteristics, 

2. Specify system effectiveness, ?nd 

3. Specify system resource requirements. 

This report summarizes the work accomplished to date in implementing the 
cost-effectiveness method developed! and documented in NEL report ! 323. The 
method as develop°d to dale was applied to a real communicatitn situation to 
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determine its validity. An hf Fleet broadcast link was selected because of the 
availability of data pertaining to hf equipments and other studies concerning this 
type of link. The performance model was used to evaluate and analyze an hf 
Fleet broadcast link with respect to performance and cost. The analysis illus- 
trates the contribution of various system parameters to overall system perfor- 
mance. A particular geographical hf link was selected for the analysis to provide 
propagation losses, noise levels, operating frequency, and antenna gain as 
functions of radiation path. 

Primary effort to date has been di.ected to the system performance model 
and its cost analysis. During the course of this work, it became obvious that 
the parameter "cost" as used in cost effectiveness should be more general and 
should include, in addition to dollar cost, such items as material, time, and 
personnel. The more general descriptor "resource" will be used in place of 
"cost" in order to consider these items in proper perspective. In most cases, 
dollars will still be involved in the analysis. However, the limitation of asso- 
ciating everything with dollars is removed. 


METHODS 


Problems in Cost- Ef f metivenoss Analysis 

The following comments on techniques for the analysis of cost effective- 
ness draw from the content of related references 2 through 5 as well as from 
experience gained during the course of this invest igation. The uninitiated are 
likely to think of cost effectiveness as a method that maximizes effectiveness 
while minimizing cost. This conception is overly optimistic. What cost- 
effectiveness methods can do is to minimize cost of a system while holding 
effectiveness to some minimum acceptable level or to maximize effectiveness 
while holding cost to some maximim acceptable level. The possibility arises 
that the analysis will be misleading if the wrong fixed levels are chosen for 
cost or effectiveness. A way around this difficulty is to compare the results of 
analysis at several levels of cost and/or effectiveness. 

The optimization should be exerted for the system as a whole. If the 
subsystems are optimized individually ("suboptimization"), the result will 
most likely not be true optimization. For instance, the best receiver for some 
given cost might have high sensitivity and low stability, but the best system 
nh^ii call for a receiver with moderately good values of each. 


PROBABILITY LIMITATIONS 

The single-link problem can be express.'** in terms of physical parameters 
rather than probabilistic terms. This method avoids the problems Associated 
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with the use of unverified probabilities. Solutions to the problems associated 
with unverified probabilities will be sought during subsequent investigations. 

An example of an unverified probability is in the answer to this question: 

What is die probability that two ships will be transmitting on the same frequency 
simultaneously? Sometimes questions tike this one cannot be avoided. If 
they cannot, the sensitivity of the analysis to die assumed probability should 
be tested by analyzing the system for various values of the probability under 
consideration. If the analysis turns out to be sensitive to the probability, the 
results should be considered skeptically. One tempting way out of the 
un veri f ied-probabi I i ty impasse is to accept the word of some outside authority 
on the matter; but this ^s merely a way of avoiding the chore of reconciling tine 
analysis to this problem, since it is obvious that if the probability is unverifiabie, 
the expeit has no more assurance of its correctness than the analyst does. 

Again, in treating matters of probability, the analyst must realize that the 
most likely event dees not always happen. Low-probability eventualities should 
be inspected, too, for disastrous outcomes. When the real situation becomes so 
complex that it cannot be analyzed directly, a simplified model is used, with 
the danger that the analyst will become more interested in the model than in the 
real situation. Game theory is deplored by many because it is so often invoked 
for simple models, but is so difficult to apply to complex reaHife situations. 


DATA LIMITATIONS 

A pioblem which this study encountered was the limited cost data avail- 
able. This is a warning that the cost equations obtained by the regression 
analysis will not be as accurate as desirable. Under these circumstances it is 
especially dangerous to extrapolate the results to ranges of the variables for 
which there a,e no data. 

In the same vein, it is a mistake to ignore a variable which cannot be 
quantized; for example, ease of operation of equipment. It is always easier for 
the analyst to insert the effect ol a quantifiable variable into a performance 
equation than to philosophize upon the effect of an unquantihabie one, but the 
tendency to ignore unquantifiable factors should be controlled. 

When the present study is expanded to include more complex prob'ems of 
evaluating large comr.uniution systems to be built in the fulitfe, the costing 
methods will hav- to be modified and extended. Some of the considerations which 
will become important are the period over wh'Ch the cost of new equipment is 
amortized, the difference between sunk costs (money already 3pent) and future 
■:os*s, the significance of cost differences, and ways to attach dollar costs to 
tra'mng of personnel and other expenses not directly for equipment 


LIMITATIONS OF SINGLE-PARAMETER ANALYSIS 

Most systems arc not so simple that their effectiveness cm be expresses 
as a single parameter If they were, ^ost-e flectivcness -ualyscs would be much 
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■ore straightforward than they ait. For example, if the effectiveness of a 
coMamicatioB system wete equated with information tate, ail considerations of 
reliability and maintainability would be sacrificed. 

Combining die effectiveness measures of s system into a single “quality 
factor" is also specious. For s time, this w*§ a popular practice, but now it is 
discouraged by many practiced analysts. Appendix A* illustrates one problem 
associated with defining effectiveness as k single measure. A particularly 
strong statement on the subject was made by E. S. Quade:* “One thing we 
cannot do is construct from a!l die individual objectives some group objective 
by appropriately writing nil separate ones; this is a practice! absurdity and it 
has been theoretically demonstrated that there is no w que bid satisfying way 
to do it." For example, consider two systems with pertbrmsnce factors, A, B, 
and C of equal importance (weight) evaluated on a scale from \ to 5: 


Factor 

A 

B 

C 

Sum 

Product 


System 1 

4 

1 

5 

10 

20 

Ql 

System 11 

3 

4 

2 

Q 

24 

Oh 


If the rality factor Q = A ♦ B ♦ C, system 1 has the hitter Q- if 0 = A * B * C, 
system H does. And there is no way of deciding which formula for Q (if either) 
is legitimate. Furthermore, deciding upon the relative impuunce of the perfor- 
nance factors is an arbitrary process with serious consequences. If the pro- 
duct form of Q is used, weighting does not affect the ratio Q ( /Qu- However, 
when the sum form of Q is used, giving factor B a relative weight of 2 gives 
system Q the advantage of 13 to 11. Likewise, system l has a IS to 11 advan- 
tage if factor C is given a relative weight of 2. Finally, the evaluations are 
sensitive to samll changes in the values of the performance factors. If these 
are not specifically quantifiable, the dsnger of upsetting the resells is great; 
for example, if Uaoi B of system I is changed from 1 to 2, the product Q is 
doubled and far outstrips the comparable Q of system 11. 

After all die arbitrary manipulations are made, the probability is sur- 
prisingly high that the (product) Q factors of the t.vo systems will he equal, as 
shown in Appendix A. 


Systtn Effaetivenass Modal 

If the description of s hypothetical communication situation is given, 
system requirements can be ^ecified. The lequirements determine the objec- 
tives of the systea Communication system objectives xay be amny and varten . 
Some typical communication -system objectives are: 

I. Information reliability 

2- information rate 

V System reliability 

4. System availability 

V Anti-DF 

6. Anti-jam 
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This report considers only the first two objectives. Subsequent work will inte- 
grate other objectives into the overall Model. System objectives may in nasty 
cases be considered as measures of system effectiveness. Cost as a resource 
element is not considered a measure of effectiveness. Cost is a criterion for 
choosing between alternative systems at some specified effectiveness level. 

The RESULTS section iilusttates how cost can also be used to select between 
alternative systems that exceed the minimum r equ i r em en ts to different degrees. 

Figure 1 illustrates the procedure followed hi evaluating system resource 
requirements and effectiveness. 

The resource-effectiveness analysis begins with a given act of system 
requirements ss shown in figure 1. Finn the requi rements , mission objectives 
are specified. The mission objectives indicate *he type ef optimization to be 
sought; that is, the swat effective system for a given level of fading or the 
least expensive system for a specified level of effectiveness. The type of 
objective is also reflected in the constuiats imposed on the optimization pro- 
cedure. The system model is sn analytical model of the system intended to ful- 
fill the stated objectives. The system model iatcrrelates system characteristic? 


REQUIREMENTS 6 
EFFECTIVENESS 
OF DESIGN 
ALTERNATIVES 



f'0*r+ I. »»W m unolftit. 
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and misaioa objectives, to mm instances the system modti may be modified 
to evaluate as alternative solution to the Mission objectives. This alternative 
could require a entirely sew systesi model. System characteristics are cate* 
gained by ssteodela; each submodel symbolises a particalar type of coamuni- 
cation equipment. The resource interface aodel (cost analysis) assigns cost 
factors to system characteristics within each submodel. The submodel cost 
expressions ate then used in the optimisation program as either criterion or 
constraint expressions, the use depending upon die type of optimisation being 
performed. 

The optimisation procedure provides the analysis outputs, an analysis of 
tunfcivfey of cost or effecrivcaeM to changes in system characteristics in addi- 
tion to the optimised system cost or effectiveness configuration. 

It RESULTS system characteristics are shown against com to indicate the 
type of trade-off analysis possible from the method. 

In the previously developed method,! the coat (rename*) effectiveness 
of a c.wwmi cation :yg u* it* Jaded system availability and system reliability 
in addition to system per torn* ace as measu r es of system effectiveness. Avail- 
able time has precluded the in elusion of system availability and system relia- 
bility in this report. Refer to appendix F for explanations of the symbols used 
throughout the report. 


Ptrftrumt 

The effectiveness model which evaluates the performance of an hf shore- 
to-stap communication link consists of severs! sa hr .odds. Each submodel 
serves as tm interface betweea subsystem characteristics and subsystem cost. 
The submodels are: 

1. Transmitting terminal 

2. Transmitting 

1 . Tmirminiag Vienna coupler 

4. Transmitting antenna 

V Recct . tag antenna 

6. Receiving sntama coupler 

7. Receiver 

g. Receiving terminal 

Execution of the method depends upon the ability to ass ki ate s cost 
factor with a system performance factor. The procedure fa detirmuimg cost 
facias is described m Com Analysis The procedure was used to determine 
the cost factors listed » table I. These items were shown to be critical in 
the establishment of con« nd performance factors. 

Subsystem performance facias art those am/or subsystem characteristics 
that directly influence the attainment of the systems objectives. The system 
performance fee: ~s me combined by analytical expressions according to cos- 
mmmicstfon theory to yield the measures of system performance. The performance 
an defined here consists of informs non rehrJnUty and information rate System 
cost met ore modified hy system performance factors are combin'd to give system 
cost for die predicted level or performance. 
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TABLE |. SUBMODEL COST AND PERFORMANCE FACTORS. 


Performance 

Factor 

li 

Submodel 

Coat 

Factor 

Number of channel* 

Bit duration 

Type modulation 

Transmitting 

terminal 

Number of channels 
Quantity of units 

Average power 

Stability 

rransmitter 

Average power 
Frequency 

StaF »ity 

VSWR 

Transmitting 

antenna 

courier 

VSWR 

Frequency 

Number of inputs 

Gain 

An'ctma pattern 

Antenna orientatk*i 

Transmitting 
a: teuna 

Cost pet antenna 

Gain 

Antenna pattern 

Antenna orientation 

Receiving 
antenna * 

Coat per antenna 

Norse figute 

Receiving 

antenna 

conpler 

Noise figure 

Sensitivity 

Stability 

Receiver 

Sensitivity 

Stability 

Frequency 

Quantity of units 

Number of ctunneis 

Bit duration 
fNpe modulation 

Rece ving 
terminal 

Number of channels 
Quantity of jnits 


lb* R.inuMM»-«: 03 i system may bt defined for a specific level of perfpe- 
Miur by utilizing opi imitation ptucrdmev Mao, the — a imnm-pct foraance 
svsten may be defined for a fived investment. 


PROPAGATION MEDIUM 

Pvdpagaiwn-fardiction fuograms peo'.ide a method of amnlatatf analytt- 
calli thme geographical and emtreow octal factor. March affect a sr»»rm’i 
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effecti resets. Then «re smral cm pa ten zed propagation-prediction prograas 
available to die utlyii. The capabilities sod liaitationa of the vary. 

A astvey of pcopafatwa-predictioa prograas known and available at NEL ia 
contained hi appeadix EL It is this type of empale? propaa that will be used 
ia propagation Modeling. 

For rilia aaalyais, the NEL Hi^ Frequency Radio Piopagatioa Cm pate? 
Prograa* was wed for MOdelisg the propagation Medina. The preyaa caka* 
laics the following characteristics of the propagation pathfs) as a Inaction of 
(he thae of day, aoadi. tmpot amber, tnasaitter location, aad receiver 
locatioa. 

1. Noise voltage (receive?! 

2. Propagation loajea 

3. Skywave aodc (amber of hops, teflectia* layer) 

4. Aatcasa gaia as de tenoned by tafce-off aagle aad angle of areival 
The prograa also pern its vanoas aatcaaa types and pattens to be coasideted 
hi the calcaiaiioaa. 

All calcslatioas ia this report were baaed oa a shore taamitte? located 
ia Koaolaia, aad a daphoacd receiver located at vatteea distances along a 00 
azusafe frm Koaolaia 

The otho characteriatics of the ccmsoaications link are: 

Swnapot amber 20 

Month December 

Host 0600 


INFORMATION RELIABILITY 

A mjor objective hi aay cmaaeicattoas syatea ia die reliable transfer 
of mforaatioa. kaforaathn reliability May be areasored ia arretal ways; how- 
e-rer, each Method is dependent apoa the received energy per bit aad die noise 
power deasity at the receiver. Tic man c o mm on are awes of isforwatwa 
reliability are bit error rate (BER). character error rate, and word error rate. 

Each Mtaswre of jifanatioc reliahr’.M is a twrttoa oi the BER. "fetch ia 
depended apoa die normalised 5/S ratio (R). 

The can cm empntensed Model lot reformation reliability does aot convert 
the probability of bn error (P,) to dre coweap oo d iag nooaatued f'N ratio 
This coo vers wo is Made owistdt die prof am aad is selected frm fee appro- 
priate equation listed ia reference (iy Eqsuwo (I) relates fee 5/N imtio ia- 
provcMeal possible ta the detCkim Model, ho the predetea wn S./N mtto 
(RPRE) to fee postoeteclKM S/N rstw The asalys» of FSK detection 

■Inch Allows a for t aoahsear receiver wife a s-agk biter tod haeai djvrrusi- 
aaecr 


RPRE - t l »« -ill a h 

*0 


i-u 
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5,,/No = RT 0 B, ( 

f w sB,/2*l/(4V 
Bj 3 3/(21^) 

(&if)L m ®V* 204 ®| 


RPRE* c* *m 


V(4T fl ) r 3RT fl 
0.B4 (20 ♦ 1/(2T 0 )) 2(T 0 - 2 ) 


(1> 


RPRE »» a fcmctkm of mofcla ri o a type, receiver Sheriff, post ieafth, 
gaard time, and aaket of rhaanr’i 

Eqaatk* (2) equates RPRC to 6c propagation mediae ckmcMriakt mi 
perfomsace parametera of t Be qtita submodels. 


r # 

)0 log t0 fRPRE) 3 10 lo^p, ♦ 10 ♦ G,4 G, ♦ I0k»l w 


( 2 ) 


-L'-F-Cf- 10 k>8 t0 [ 1 ♦f Me/ .fS r KTB 


where 

F. = E , - » k, [0 f*,, + 65.5 HO 6 

F 10 | 

K * ^ = 134 - 10 log 10 0 

The following propagation betas ue obu+ned as uatpat data from the 
SEL propagatwa-predicttoii props* « 


Propagation lows 


Tract anil aateana pia 

(C.) 

Receive otou pia 

(G,) 

Effective aatouu aoise 

(£,) 

Freqacacy of operatic* 

(fttc/j) 

I 

£ 

a 

factors permit a system's performance to be 


analysed far a specific geographical locauon tne. sed map 


INFORMATION RATc 

hi be design of coaewNtoo lyatM. be re are certain trade -off* that 
can be made b e t ween nfoimaiica reliability and aforawnco Rie A compote* 
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wbroutiae ku beta written that enables these trade-offs to be taken into 
ac coaat. Exacts of Ike trade offs possible are as follows: 

Ike greater tke darstioa (stow rate) of tke pstse clement, the pester the 
potato lit y (high teliatotltty ) of receiving ike element correctly. Tht converse 
is also trse. Also, Ike an errors there are is a aessage, the less mfonsatioo 
can be tr—sfentd boa scarce to saer. 

Wbes two systems Aat aae different coding techniques are compared, a 
detailed analysis is leonaed to dctcmiae the rate at which « format ton is 
to— aafrurit. ia teletype systems, the start and stop bits, as veil as error detec- 
taa and correction bits, mast be considered tr the detensiasuaa of mfons.tk» 
rate. When two systems tel aae the same coding techmqae ate compared, it 
ia not difficult to determine their relative tn format ion rates. However, coding 
techs tqaes and iafoosattoa rate want be related to a am conaaon factor for in- 
c o rpot a tioa widbn the method. This cons mo fector is gives by equation (31 
Egsattaa (3) stales that the channel information talc ia eqas! to the scarce rate 
redaced by the channel apivocatioa 

l*C s Hft) - tyr) (3) 

I RC = Rate at wh di ta form at ton is transferred 
H(i)= Source trmtanisstrw me 

H y (r) = Equivocation * 

The soacce information rate (HJ is modified to considt both sterf, and 
stop bits as well as enor-de'ection bits Kj facilitate calculations. Tlu* is,' the 
same data rate is redaced by the effect of start and stop bus sod uroi -detection 
bits. In die Ena! measare ot iaionaation transferred tv the wonrcc, information 
rate is further reddeed by the channel equivocation. For the binary system th. 
channel informal' on me is 

Irc ^[fW,.log 1 (l-lv*r,k* 8 P f | (4) 

The overall system inform*: ten rate is directly proportional to the number of 
channels c sad inverse!! proportional to the order of diversity m as tpven in 
etjaatkr (5) 


lata last 


One of >br p so b fems associated with cost-effectiveness enalvs’c is the 
toi lection of reliable tola. Standardized sources of system char at (eristics and 
cast ue definitely lacktag Steps have been taken to esubitsh • data straps 
and retrieval svstt* at NEL »o sappart the resewee effectiveness proyraT This 
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data base will be continually undated to pto^id* a sliding historical data base. 

The storage ar.a retrieval program will provide data from which cost factor, can 
be determined, in order for cost effectiveness to become a continuing part cf 
the Nrtvy procurement process, a complete data base of Navy equipment is 
required. The be''* time to acquire equipment dam is in the initial stages of a 
system’s life cycle. The types of data that are .squired to support reflomce-effecrivc- 
nesa analysis are as follows: 

1. Standardized listing of equipment characteristics by type «4 equipment 

2. MTCF 

a. Specification value 

b. Predicted value 

c. Actual value 

3. M1TR 

a. Specification value 

b. Predicted vau u e 

c. Actual value 

4. Unit cost 

a Quantity procured 

b Date procured 

c. Spares 

5- Training cost 

6. Installation cost 

7. Personnel requirement 

8. Development costs 

The data sources used to support sue resource-effectiveness, analysis of 
the iif link contained in this .-eptm are listed in append:?: C with typ of data. 


( 


Cost Analysis 

Hie cost analysis applied here assies a cost factor to each specific 
equipment characteristic so that equipmen* costs :an be predicted as a function 
of equipment characteristics. A computerized statistical multiple-regression pro- 
gram is used n determine each cost function (tom historical cost data and equip- 
ment characteristics. A brief explanation of the theory and analytical process 
involved in curve fitting via multiple regression is contained in appendix D * 

Total equipment costs can be considered to consist of fixed costs and 
variable costs Fixed costs are independent of the performance expressions 
( METHODS, equations 2 aod 5). Variable costs vary with level of performance. 
Fixed costs here also includ? performance factors that fas! to correlate with cosi. 

Total cost - fixed cost + variable cost. 

On the first try at determining a cost expression for a particular type of 
equipment - for example, a receiver - all equipment characteristics (all on 
which there are sufficient data) are submitted to the icgression program at once. 

If the equipment cost fails t> correlate with the pertinent equipment character- 
istics, the equipments arc further categorized by frequency range, type ostalla- 
non. or other differences. Graph plotting of equipment characteristics versus 
cost may be employed to help determine the snalytical form of the cost expres- 
sion. 
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to some instances the equipment characteristics required to evaluate a 
system's effectiveness tray not correlate with equipment cost. Such pertinent 
equipment characteristics as modulation t; pes, number of types of modulation, 
transistorized equipment, number of channels and isolation did not correlate at 
a significant level with equipment cost. It is not implied that they will not 
correlate, but only that in Tarnation in extreme depth was not available. Also, 
equipment characteristics may correlate negatively with cost (that is, receiver 
cost decreases with a decrease in sensitivity!. In lower-cost equipments 
($3000 or less) the quantity of units procured affects the unit cost significantly 
to this price area, as well as with the more expensive equipments, the quantity 
of units procured is considered when this type of information is available. 

If the data on a particular type of equipment are insufficient to permit 
curve fitting, the equipment should be treated as a discrete entry in the perfor- 
mance and cost equations. This approach was taken in the following analysis 
with respect to shipboard and shore antennas. 

The computer program used in the curve-fittins process provides several 
statistical tests to evaluate the "goodness" of the lined curve. These tests 
are the c test and F ratio test that are described in appendix P. The multiple' 
correlation coefficient and standard error of estimate are also calculated. 


gpttoizatiii Ttchiiqve 

One of the main tools of the resource-effectiveness method is the Systems 
Optimization Program (SOP). 7 Several minor modifications have been made to 
the SOP, some as adaptations to the current problem, and some for compatibility 
wiJr NEL computing equipment. The SOP minimizes a given function, called 
the criterion, while satisfying two types of constraints. The constraints can be 
in the form of equations or bounds on the individual variables. These constraints 
are always satisfied during the optimization procedure. 

to connect i or, with the present evaluations of communication systems, the 
cost is written as a function of system parameters, and this expression becomes 
the criterion equation in the SOP- Only one constraint equation is used; in it 
the gatos (power, antenna gain) are balanced against the losses (path loss, 
noi**\ 5/N ratio). Some of the "unables are also constrained within preset 
limits. 

The roles of the cost and gain-loss equations can be interchanged; that 
is, the cost can be held less than or equal to a certain amount. The gain-loss 
equation can be used as the criterion by writing S/N ratio in terms of the gains 
said losses. Then the SOP will maximize S/N ratio by minimizing its negative. 

The SOP has four major subroutines, called Mode 1, Mode 2, Mode 3, and 
Mode 5. Mode l is the most important, as it executes the optimization. The 
technique used is the method of steepest descent, modified to work with con- 
straints (see appendix E). The cost and criterion equations are written as 
functions of the system parameters. Some of the parameters are variables and 
may be perturbed in the process of minimizing the criterion. These parameters 
currently arc transmitter coupler VSWR, transmitter and receiver stability, 
receiver sensitivity, transmitter power, and receiver coupler VSWR. A greater 
number of the system parameters are held fixed, but may be changed for each 
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run of ;he program. They include transmitter operation time, nuaber of tran«- 
miners purchased, path loss, and environmental noise. 

Mode 2 evaluates chosen functions and their derivatives while varying 
one parameter. It can also plot these results, a capability which allows the 
analyst a convenient way to judge the effects of individual parameters upon the 
complete system. 

Mode 3 evaluates chosen functions while varying se”?ral parameters simul- 
taneously. 

Mode 5 is a sophisticated output routine drat lists the results of Mode 1 
in a complete and readable form. It can also convert units from those convenient 
for calculation in Mode 1 to those appealing to persons using the results. 


Gonfidsnce Level of Predicted Performance 

The performance equation contains, or is dependent upon, several para- 
meters fc: which one cannot specify a “true 1 " value but only the most likely 
value. Hence, these parameters are represented in the performance equations 
by their most Likely, or mean, values. A statistical confidence factor is used 
to compensate for the effect of parameter uncertainty upon system performance, 
thereby assuring a specified level of performance with a given degree of confi- 
dence. The confidence factor is included in the performance expression (METHODS, 
equation 2; as additional system loss. The confidence factor is determined 
from system parameter uncertainties 8 and the desired level of confidence. 

The uncertainties are: 

°sig ~ uncertainty in predicting signal strength over ionospheric path. 

a TA = noise variation about the mean. 

°ant = uncertainty in receiver antenna gain due to receiver antenna 
characteristics. 

a Fa = uncertainty in mean value of noise. 

°nf = uncertainty in receiver noise figure value. 

The total uncertainty o T is 

f i V2 

°T - | °ANT 2 * a SlG 2 * a TA 2 * °Fa + a NF j 

Confidence factor (C^,) - Ka T 


Confidence Interval, Percent K 


50 0 

90 1,282 

95 1.645 

99.9 3.09 


The confidence factor ur.:d in these calculations were obtained from the 
following system uncertainties. 

° SIG = 8 dB 
a A v t = 8 dB 
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°TA s 5 dB 
Opa ~ 5 dB 

°b'F ~ ^ 

ITjc total system uncertainty {a?) is found to be 13.675 dB. The confi- 
dence factor for a 95-percent confidence is then: 


C f -Ka T ~ 1-645 x 13.675 dB = 22.495 dB 


RESULTS 


The results of analysis of hf-link effectiveness are presented in various 
forms to illustrate the capability and versatility of the method sj far developed 
under this program. Performance and cost equations utilized in the optimiza- 
tion program are discu&sed with respect to contributing factors and their inter- 
relationships. Depending upon the form of the optimization process, the per- 
formance equation may be either the criterion or the constraint equation. Cor- 
respondingly, the cost equation will then be the constraint or the criterion 
equation. 

In the optimization program a local minimum is sometimes found rather 
than the global minimum. In this case the starting values for the variable para- 
meters are changed to determine if die optimization process can locate a new 
minimum for the criterion expression. 

The optimization program uses 26 parameters (fig. 2), of which only six 
are allowed to vary (variable parameters). Other parameters could be selected 
as variable parameters; however the six selected (fig. 2B) are believed to be 
the most significant with respect tv system cost-performance trade-offs. Figure 
2 is a typical SOP output page. It was taken from a computer run in which system 
cost, parameter 12 , was used as a constraint on system effectiveness. 


parah parameter description pjxeo units 

number value 
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Sikuidtl Gist Pridietiii 

The aethod of analysts required that cost expressions be generated for 
each submodel of the coamwnicatioas system. The expressions were developed 
fo all but the two antenna systems, shipboard receiving and shore transmitting. 

The data available on these types of antennas were insafficieat in permit cost 
expressions to be developed by regression analysis. The costs and character- 
istics used for the antennas were takes horn vendor literature. 

The coat expression for each submodel along with the multiple-correlation 
coefficient and standard error uf cost estimate {a) are included in the summary 
that follows. The mnge for etch parameter used in the regression analysis is 
also indicated. 

The symbol Q is used to denote where applicable the quantity of miits to 
be considered in predicting the cost of a submodel. Each or the submodel coat- 
prediction exp-essiuns was generated by means of the techniques described to 
the Cost Analysis section of METHODS- for an explanation of the symbols 
used in the cost expressitr., see appendix F* 

Consider a typical example of unit-cost prediction using (be following 
receiver characteristics and the receiver cost-prediction expression (METHODS, 
equation J). 

Receiver Characteristics 

Q = tO = Quantity of units to be procured 
F 0 = 16-250 Mc/s = Receiver center frequency 
S n s l. V s Receiver sensitivity 
S r = I PPM = Receiver stability 

Receiver cost ($) = ♦ $5,)8;.9l6 - $2.48269 (JO) - $659.2069 log 10 ( 16- 25* 10*) 
fixed c »st variable cost variable cost 
due to quantity due to frequency 

- $1,786.09)1 1 log j 0 (10 x |.$)|+ $2,016.81)87 log, 0 (100/1) 

variable cost variable cost 

due to sensitivity dec »o stability 

Receiver Cost <$) = $2,481 when bought in quantities of 30. 

The tange for erch variable used u the receiver cost analysts is ss 
follows: 

14. Qi 2,217 Quantity 

22 i 32).375kc s Frequency 

I ^ Si i 10ui‘ Senstttv.ty 

01 * S f vt PPM Stability 

Red ic ted costs can be obtained for the other equipments comprising the 
svsiem from ihe appropriate submodel equation m die summary which follows 
Die coefficients assigned each cost factor are the performance factors and are 
determined bv <hc opoatzat; « program The total hf system coat is (be mm 
of the individual submodel costs 
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SMUuvy of Mbaodel coat prediction (qutiou. 

1. Temiual Equipaeut 

« Receive or Trasasut Tcmisal Cost ($) 

* -943.239354 ♦ 1091.5^3361 (c) 

-69.069296 (Q) 

1*Q* 29 
1 £ c * 16 

Multiple Carte (ado* 0.99909 
9 * 56354(389 

b. Receive mi ToMBit Temtaal Co« ($) 

* 4s'3. 2J9336 ♦ 2163.090722 (c) 

-136.17552(0) 

UQ*5 
U <*16 

Multiple Cone let »oa 0.99909 
a - 5635.46589 
1 Receiver Couplet Cor. ,5) ^ 

3014.239575 - 746 *66745 (F e ) ♦ 85.11956 <F e ) 2 - 3441748 (F e ) s 
fc F e * 15 

Multiple ConeUtkM 0.99005 
a - 540 799290 

3. Receiver Cost <$) = 5,382.916 
-2.486260 (Q) 

*659.2069 lof iofo 
-1786.0931 {log, odOS.)! 

*2016.81387 log, 0(100/5^ 

34* Q* 2.217 
22* in £ 323,3758*. 

13,* 10 
001*5,* l PPM 
Multiple Correlation 0.980429 
a - 5568 48309! 

4. Tt»«sitter Cost (5)s -154.942.190818 
♦7216.0810 tog I0 p ( 

♦1 7039107 
♦71683.2036 (5>*/f 0 ) 

♦9205.5072 log, ^ 100/S,) 

0.01* $,* 1 0PPM 
♦2 * f kM * 28 Mc/a 
13 * |q * 16 Me/s 
300* ft 20.1T5) w«!U 
Multiple Correlation 0 997445 
9 ~ SI 052 .209776 

3. Tranmit Couple? Cost 3 - 
14.957.591461 

-2057 4931 «e» l0 fo-3« 1224 V, 

♦1188 842182 (, 

1 4*v,s 2 0; l 
1 3-15i fo* 312 5Mc/» 


w a *. 
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c f S 4 

Multiple Correlation 0.913099 

a = $249.653032 
6. AntaHns 

a. Transmit Asieana * Conical Monopoie Cost ($) = $*.275 

b. Receive Antenna * Verticil Ptoip Cost ($) = $100 

Tki Effect if Prisary-Piwir HigiiriRiits 
llpii Systin Pisiii 

Table 2 illustrates (be cost of ao hf communication link as a haictioa 
of a receiver removed fro* ibe transmitter is a fixed direction with mgt as a 
vsnabie porasieter. The variables that affect tbe sysieat cost ate frequency of 
operatioa f^ /t . external noise power (E„) available at tbe antes ca. and pro- 
pagation loss (L,). Ibe values for these vshabiea ax detemiaed by ike pro- 
pagation-prediction program for each specific iwagc. Also, the frequency of 
operation selected is the optimum frequency for that range and receives torn turn. 
Ibe S.rN ratio a> a specific receiving location is a function of the effective 
•tana noise figure and transmission pain iota. Tbe received noise field 
strength and operating frequency cause tbe effective antenna nor sc figure to 
vary irregularly with range. Tbe variation causes system price b liactnate 
along the selected path rather than being noootouic with distance. Tbe first 
three line entries in tsbie 2 illustrate ibis situation. Tbe path luss (L p ) is 
■onotontc with range, increasing from 64.7 d8 at 240 a.a. to 74 7 dB at 960 
n.m. However, the effective none at the receiver ar'ona (E„) decreases from 
14-3 (iv at 240 n.m. to 3-4 pv at 960 n.m. This decrease in E* combated with 
an increase in operating frequency mure than compensates for the 10-dB in- 
crease in path loss, resulting in a less expensive system fa 960 n m. than for 
2«0 n m. 

In the last two columns the effect of peiaoty power (a commercial power) 
is shown. The primary -power cost for a Vyeai period wus included m the cost 
(cnterion) equation. The equipment was considered to be tn operation 720 
hours p« fiunih in most cases, the effect ot considering primary power and 
its associated cost was to reduce the amount of the transmitter power output 
requited, causing >he system to seek additional gam from other system para- 
meters at less cost to tbe system. 

The expression foe penury-power cost uses the rate structure of the 
San Dicjp' Gas L Electric Co., as this informs (ion was readily available. Tbe 
amount of puma.? powrr required is dependent upon the efficiency of die trans- 
mitter. plus other items. The dtteteaer of dwee transmitters was found to 
decrease as tranv.ittei power is increased, and this relationship was included 
in the expression fra primary-power cost. Tbe effect of transmitter efficiency 
on S' stem cost breomex significant as the communication link range is increased 
i row * .9*0 n m to > (MO >i m. The communication link ana fy led consisted of 
e»chi digital cbawwcls each operating with a bn err j» rate f B£R| of ltT 3 errors 
pa bn. f be link wj\ orttuiied toe each rarre considered. The confidence 
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TABU 2- 

ST STEM COST AND POWER REQUIREMENTS. 

E„ 

fuc/i Withont TTiiaary* 

| With Primary* 

mV 

Power Costs 

Power Cost 3 



Power, 

Watts 


level associated with the error rate ia 9) percent. The BtR ms determined 
ior non coker eat FSX sndalatioa anbfected to Rayleigh fadui* and Gaussian 
wise, la addition to nanmaer power, the optimization prop** varied other 
aywec characteristics. The other system variables that were permitted to vary 
ate: 

1 Transmitter stability 

2 Receiver stability 

3 Receiver sensitivity 

4 Receiver coapler aoiae figure 

5 Transmitter couple- V51 R 

Becsase of the nttncim ut variables L,. E, . «*d f Ur/t . ih« worst* 
case confi^KatMM sty not occar at the nariamai range over which comnmnica- 
twns at desired. tatenaediaie napes, gee graphic a! locations, and tune 
shonldall be considered in aetersuatag the destpi mjuirrmcats or the adequacy 
of s enn — m eat nan system. Also of laporttace ia the Oernp of a s. ? stem is 
the tras mutter efficiency with its associated pnawrypewer cost. 
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SystsRt Cost Varsas Rani* far Variant Valais 

off. 

Figure 3 illustrates the effect of cbJUMBicatne'link range on system 
cost The curves are for effective antenna noise figures (F a ) of 40, 30, and 60 
dB. The link performance for these calculations is constrained to an eight* 
channel system operating with a BER of (O' 3 errors per bit with a 95-perceat 
confidence factor. Each point plotted in figure 3 represeats an optimized 
(minimum cost) system. 



200 JOQ 1.000 2,000 5,000 


RA*C£ fNAUTlCAl. MILES' 
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Specific coasnmicatioa-Uiik characteristics we: 


Link location 
Modulation 
Sunspot nrnmbct 
Month 
How 

Transmit AiteaM 
Receive Antenna 


0* azimuth bon Honolulu. Hawaii 
Noncoherent FSK 
20 

December 

0600 

Conical Monopole 
Vertical Blip 


He curves illustrate the mat increase associated with a 10-dB increase 
is the effective antenna noise figm*. For a 2.000-n.m. link, an increase is the 
effective as touts noise Rgme km 50 dB » 60 dB would require an additional 
$21,000 investment [19 percent too rase) in the system. 


Systta Cist V»:sis HR is t Fnnctiin if F a 


Fignre 4 is a graph of System Cost versus BER as a fimctton of F a 
(effective antenna noise figured for values of 40. 50. and 60 dB- Each puut 
plottej represents ar optimized system (drat is, minimum cost) evaluated to 
provide a specified level of fcformancc. ill calculations are for a fired range 
» dim teats variation in S/S ratio due to pate loss and receiver kvitmt 
The characteristics of the Hah are as follows: 


Range 

Confidence 

Channels 

Modulation 

Location 

Sunspot number 

Month 

u~_.- 


3.000 n.m. 

95 percent 

S 

Noncoherent FSK 
0* azimuth (Hawaii) 
20 

December 

>1600 


The graph illustrates the cost associated with designing a system for 
different effective antenna ooise figmes The curves indicate drat fra • very 
slight increase :a system coat the B£R can hr improved horn 8 * HT 4 ’m5« 10-* 
«*<*» per hit. 

Tfcs«t type cf curve can he anrd to compare the BER imperwerveni widi 
cod^ag *uh thai o ba turd horn a different system design. 

Figure 5 i! terms tea the tm{acv<meat is information reliability obtainable 
with an increase in system coat. The calceSaiwn* have been srade for pad 
fosses of 10. 10, tad 90 dB. The effective anterma noise figure used « the 
optimization cakateriom san v ousuatnrd at •%) JR ,V oprnaam frequency of 
opera! inn was dr term toed by the ht propagation - prediction prc^nrai, and was 
uued in each cakuteinn for the range involved hi this type of optutiraiioa, 
coat is 2 conatramt and (^format «>a reliability is the entenon 
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10-1 10-2 10-3 10-4 10-5 


BER 


Figure 4. System cost versus BER as o function of F a . 
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TOTAL SYSTEM COST (DOLLARS) 


Figure 5. System cost v rsus F e for various path /osses. 
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Figure 6 shows receiver cost versus receiver center frequency for three 
values of sensitivity. The average talk) of receiver range to center frequency 
was 1.58- The ratio seems to hold in most cases regardless cf receiver ceuter 
frequency . 

Figure 1 illustrates the relationship of hf transmitter cost and average 
transmitter power output. The curves have been plotted for three levels of 
transmitter stability. 



RECEIVER CENTER FREQUENCY (MEGAOCLES) 


Figum 6 . Recoivar cost vrsus contor (roquoncy for various voluot of rocoivot 
sens itivity. 
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Figaro 7. Hf shoro iransmiltor 
of transmitt or inability. 


co*f vorsus porror for various vo lust 
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Sensitivity Analysis 

The hf link configuration was evaluated to determine the sensitivity of 
system cost to each of the six variable parameters. The system was first 
optimized in terms of cost (minimum cost) tn provioe an eight-chanrel link with 
each channel operating at a bit error rate of 10" 3 errors per bit. The confidence 
level over the 3,000-n.m. link is 95 percent. Figures 8 through 13 illustrate 
the effects of the variable parameters on system cost. 

The optimum system cost was determined to be 55,530 and is indicated on 
each parameter curve. Figure 9 and 12 indicate that system cost is most sensi- 
tive to the characteristics of the transmitter, specifically, transmitter stability 
asid power. The transmitter cost analysis should be reviewed to verify the cost- 
prediction expressions before finalizing the analysis. 
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Figure 6. System cost versus transmitter antenna coupler VSWR. 
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SYSTEM COST (THOUSANDS OF DOLLARS) 
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Systin CmparisiR 

The following technique provides one Method of comparing two or more 
systems with mhHiHc Measures of effectiveness. Ihe effectiveness of each 
system is determined for a specific communications link. The particular environ 
a^nia! condi t tons used for the comparison are a VOOO-n.M. tmk with an effective 
antenna noise figure of VOdB- In the example selected each measure of effective 
ness exceeds the system requ ire meets. The technique is to cost oui the mercased 
effectiveness of each measure and subtract the increased effectiveness cost 
from the total system cost. The lowest measure of effectiveness in etch area 
is used u the basis far the coct comparison 

hi this example, the cost of system l with in'oraatwn reliability of 2 c 1(T 4 
w*s (educed to the coat of system 2 with information reliability of 8x I0 -4 The 
difference in *. <iem cost is MV89I. or P-e excess performance cos; of system 
I over system 2 is $15,891. 

A stiaiiar comparison can be J..ic foe system uiubdm The resell ol 
drte ramie | the excess effectiveness cost is to normalise the performance of 
each system for comparisai. The chart oi table } indicates that system 2 is 
the belter buy of tnc two systems. 
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TABLE V SYSTEM COMPARISON. 



Systeu 

1 

System 

2 

System 

Requirements 

Performance 
bifo Reliability 

2xl0-< 

8xHH 

ixtO- 3 

info Rate 

Required 

Actual 

75 baud 

76-714 

75 baud 
76.201 

75 baud 

7605 

i 

System 

Reliability 

(MTB r ) 

1900 hrs 

2500 hrs 

1700 bn 

System 

Aval lability 

•0.997175 

•0.9978 

0.99 

! 

System cos: 

590.42: 

565,000 

i 

' 

Excess Performance 
Cost 

515,991** 

— 

1 


r- 

Excess 

nfSiaHiity LOSt 


* a rVW 

>9,UW 


! Cost of •** 
j Normalized System 

564.556 

561.000 


• MTTR = l hour 

*• Results from <17#.. 427 - $60. 556- See hgiue 4 
Normalized to the sane level of effectiveness. 


CONCLUSIONS 


». The feasibility of implementing the ircthod developed irt MEL. i c r '-’rt Hit 
las been dcmuns.ratcd. 

2 There is a drfmur need for a standardized Navy-wide data hank if cos*- 
e^cctivcftcss analysts is to cottitraic as a Navy drsi^a and management *oet. 

V Detailed costing of equipment chtraclrtist'rs such as kind of detection, 
type of modulation. and ase ( umisutt by statist icai Multiple regression may 
rvH be possible. because of other equipment ybaract eristics which mask these 
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fiaefR d wttoiiri ci ta fee repesstca analysts. His cnaay m*y ttqmnt 
the acqaisttioa of mat detailed cost dots w the development of new coxi-prtdic- 
tios tecfcaiqsen. 

4. The Models developed provide a Hut far ante detailed and comprehensive 
link analysis. 

V The Method sad tecNnqoe* developed cos be ssed is the analysts sad 
evilatii’a of hf eoj«paer>' , sad systems. They can also be ssefal ta the pro- 
pocstks of PTA, rOP sad DAP. 


ftECMMEMimOIIS 

l. Integrate eqsipacnt reliability sad Maintainability into the existing aer^od. 
2- Incorporate the ctfects of \ taring and mtcrltrenve ic fwtae relineaests 
of icaowce-effectivesess analyst* 

1 Extend the Method to iactede the coausnaicsnoas system » the oprntiOasl 
eavtroasuits. 

4. Establish a cea trained collection and distribatwa service (.meaning 
data os Navy eqnipaKats. 

V Exlead resource-effectiveness aodels to GKoapass other frequency tanges 
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APPENDIX A: A SINGLE MEASGiE OF EFFECTIVENESS 


Several recent publications have recanm endec a sii.gle measure of effec- 
tiveness, or figtae of merit, as r means of rating and comparing systems. The 
single measure of effectiveness is generally the product of several probabilities 
or rank designators that reptesem pome of the haracteristics of the systems being 
evaluated. The probabilities can represent system factors, such ?s reliability, 
system availability, or probability cf mission success. The ranking scheme 
iiivoives rating these 3ystem characteristics oil a scale of a to b (i>>c). The 
purpose of this appendix is to demonstrate some of the more basic limitations 
associated wiih satgie measures of effectiveness. 

To determine whether a single measure of effectiveness is appropriate, we 
must first look to the purpose of a resource-effectiveness analysis. An analysis 
of system effectiveness is designed to present all significant ai'ernatives in 
system configuration with their inherent ramifications of resource requirements 
and mission fulfillment in perspective. Where a single measure of system effec- 
tiveness is used, it is difficult, if not impossible, to relate trade-offs to the 
overall system objective(s), particularly in those cases in which the system 
factors are weighted on some arbitrary basis. L.arge systems with multiple 
objectives are definitely ' ore suited to the use of multiple measures of effec- 
tiveness than to the use of a single measure. Decision makers quite frequently 
request a single measure of effectiveness, as it lacrlitates decision making and 
decreases the administrative problems associated with it. A resource-effective- 
ness analysis should present an unbiased anay of possible trade-offs. It should 
not force a specific decision through previous decisions in the ccur-e of analysis 
as a single measure of effectiveness would. 

The case in which a system's major characteristics are ranked from 1 to 
? and then combined as a product into a single measure of effectiveness is 
examined in de*.iil in ?he fo 1 lowing paragraphs. 

Assume the situavion ir which system effectiveness is the product of four 
system characteristics a y, and 5 
or effectiveness -a jW 
where 


i ai ' *) 

1 <. yS. 9 

! < 5^9 


|«, /S, y, 8 are integers. 


# 

rigor.: A 1 graphically illustrates the cumulative density of available pro- 
ducts. There are 6-561 possible d.fferent ordered combinations of*, y, ?nd 8 
yielding 2 .’5 different products. Consider the situation in which a system rating 
is 3024 There are 24 different system configurations defined by the rating system 
that will provide the product of 5024- A system with effectiveness parameter 
9-8*b*7 is different from a system with parameters 6*7 ,( b8. However, whatever 
difference the systems have is masked by the single measure of effectiveness. 

!f tne effectiveness rating is 144, there are H2 possible different system con- 
figurations. 


F.-38 


14C 



>gt/r<e A?. Cumular i ve de i sity of products (singly measure of effectiveness ). 
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Table A1 indicates the number of ordered combinations that give a particular 
product. An ordered combination would consider the permutations possible vitb a 
specific combination of numbers, 

TABLE AL 4-NUMBER ORDERED COMBINATIONS AND THEIR PRODUCTS 
(Li each number £ 9) {Continued through page E-42) 


Product 

Number Otdered 
Combinations 
Giving Product 

Number Ordered 
Combinations 
Product Giving Product 

Number Ordered 
Combinations 
Product Giving Product 

1 

1 

50 

12 

■91 

84 

2 

4 

54 

52 

■si,? 

12 

3 

4 

56 

40 

180 

84 

4 


60 

60 

189 

28 

5 

4 

63 

24 

192 

88 

6 

16 

64 

44 

1% 

18 

7 

7 

70 

24 

200 

24 

8 

20 

72 

112 

210 

48 

9 

10 

75 

12 

216 

116 

10 

12 

80 

48 

224 

48 

12 

36 

81 

19 

225 

18 

14 

12 

84 

60 

240 

84 

15 

12 

90 

60 

243 

16 

16 

31 

% 

% 

245 

12 

18 

36 

98 

i2 

250 

4 

20 

24 

100 

18 

252 

84 

21 

12 

105 

24 

256 

31 

24 

64 

108 

88 

270 

60 

25 

6 

112 

48 

280 

48 

1) 

16 

120 

84 

238 

120 

28 

24 

125 

4 

294 

24 

30 

36 

126 

60 

300 

24 

32 

40 

128 

40 

315 

36 

35 

12 

135 

28 

320 

40 

36 

72 

140 

36 

324 

72 

40 

40 

' '4 

132 

336 

84 

42 

36 

147 

M 

343 

4 

45 

24 

150 

24 

350 

12 

48 

88 

160 

48 


% 

40 

6 

162 

52 

375 

4 
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# 

TABLE Al. (Continued). 


Number Ordered 
Com banal ioiJ 
Product Giving Product 

Number Ordered 
Combinations 
Product Giving Product 

Product 

Number Ordered 
Combinations 
Giving Product 

378 

60 

756 

60 

1372 

4 

384 

64 

768 

36 

1400 

12 

392 

24 

784 

18 

1440 

36 

400 

18 

800 

12 

1456 

16 

403 

24 

810 

36 

1470 

12 

420 

48 

840 

48 

1512 

52 

432 

112 

864 

76 

1536 

16 

441 

18 

875 

4 1 

1568 

12 

448 

40 

882 

24 

1575 

12 

450 

24 

896 

24 

1600 

6 

480 

SO 

900 

18 

1620 

24 

486 

36 

945 

24 

1680 

24 

490 

12 

960 

36 

1701 

12 

500 

4 

972 

36 

1715 

4 

504 

96 

980 

12 

1728 

40 

512 

20 

1000 

4 

1764 

18 

525 

12 

1008 

72 

1792 

12 

540 

60 

1024 

10 

1800 

12 

560 

36 

1029 

4 

1890 

24 

567 

24 

1050 

12 

1920 

12 

576 

88 

1080 

52 

1944 

2h 

588 

24 

1020 

24 

I960 

12 


24 

1025 

4 

2016 

36 

625 

1 

1134 

36 


6 


48 

1152 

48 

2048 

4 

640 

24 

ine 

24 

2058 

4 

648 

76 

1200 

12 

2106 

24 

672 

60 

1215 

12 

2187 

4 

675 

12 

1225 

6 

2205 

12 

686 

4 

1260 

36 

2240 

12 

700 

n 

1280 

12 

2268 

24 

72C 

72 

1296 

55 

2304 

18 

729 

10 

1323 

12 

2352 

12 

735 

!2 

1344 

36 

2401 

i 

"*50 

4 

1350 

12 

2430 

!2 


E--41 













AMCP 706-191 


i 

i 


i 

i 

S 

i 

| 

\ 

\ 


S 



E-42 


APPENDIX 8 : SURVEY OF PROPAGATION* 
PREDICTION PROGRAMS* 

litniictioR 


Uk of digits! computers for propagation calculations has grown with the 
computer development. Thus a step-at-a-tirac advance in this use has occurred 
with initial applications in the hf spectral region. No comprehensive programs 
adapted to the entire spectrum exist. Various groups have produced programs 
restricted to use in appropriate spectral regions. 

in the spectral range host vlf to microwaves the most intense effort toward 
computer solutions has been made in the hf and vlf regions. Nearly every hf 
solution in existence can trace its origin to the procedures developed at Central 
Radio Propagation Laboratory ,XRPL). The programs provide estimates of 
maximum usable frequencies (MUF). field strength, signal-to-noise ratio, and hop 
structure. The vlf programs depend generally upon solutions of the modal equa- 
tions for the earth-ionosphere wave guide. 

Less effort ha* been devoted to generating programs for frequencies above 
Hf, mostly because adequate solutions far system design are available rather 
directly by non-machine procedures. A general exception to this statement are 
the ray- trace programs, which have been adapted to machine programs extensively. 
These programs provide a picture of the energy distribution on the space illumi- 
nated by the antenna. Programs of this kind exist for hf ionospheric scatter from 
30 to 100 megacycles, and for tropospheric ducting in the microwave region. 

Most provide some kind of estimate of signal loss as well as the primary ray- 
jajec'ory output. 

Propagation-prediction programs have been generated at the following 
facilities: 

1. ITSA, ESSA, U.S. D.O.C. (CRPL) 

2. NRL 

3. Stanford University 

4. Stanford Research Institute 

5. DECO Electronics, Inc. 

6. Collins Radio Corporation 

7. Raytheon Corporation 

8. NEL 

9. AVCO Corporation 

10. RCA 

11. USRPA, Ft. Monmouth 

12. DRTE, Canada 

13. Radio Research Laboratories, Tokyo, japar. 


*TK» surv«y or K*r' ni w ** tyocifics'y All 

ky o< *K* MIL lUilU <*ky»lc* Olvtfian ui4»r t*i» tf C. M. I" s. 
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14. Radio Research Station, Slojgh, England 

15. km Prediction Service, Sydney, Australia 

In addition both DCA and NaVCOSSACT employ a wide range of computer 
prediction programs. 

Brief summaries of programs about which NEL's, Radio Physics Division 
has more than superficial knowledge follow. They are intended to do no mote 
than indicate the scope and intended use of the programs. A program -characteris- 
tic matrix that provides some comparison del. si is appended. 


HEL High-Frsquancy Propagation-Prsdictian 
Computer Program 

Radio system parameters are combined with geophysical and ionospheric 
characteristics to predict the performar.ee of high-frequency sky-wave communi- 
cation circuits. The program computes Maximum Usable Frequencies (MUF), 
probable inodes of propagation, £ layer MUF and ertoff frequencies, angles of 
arrival, ground losses, total losses, field strength, antenna gains, absorption 
losses, signal strength, noise strength, and signai-to-notse ratios. In contrast 
to the CRPL program, the program utilizes hf characteristic charts for critical 
frequencies and atmospheric noise (CCIR report 65). The solution is divided 
into two parts - an estimation of ihe field strength independent of equipment 
parameters, and an estimation of signal-to-noise ratio using antenna gains. The 
program was written for the CDC 1604 computer and is in NEL1AC 5m. There 
is an output for every operational mode, whereas the CRPL outputs only for the 
optimum mode. Computer time is less than for the CRPL program. 


AVCO Polar HF Prediction Program 

This program determines the highest and lowest frequencies available 
between two partio'ar statins as a function of time, and the geophysical and 
ionospheric parameter Propagation losses a.e determined for specific fre- 
quencies within this calculated range. The calculati on is c*lendcd to ionos- 
pheric aih disturbed conditions, bot all calculations arc valid only for high 
sunspot number The computer program is written in standard Fortran and in- 
cludes Id modes of propagation. Frequencies from l to TO Me s in intervals of 
l Me s arc toosidcred. The program prints mode, tiaiisreission angle, and 
available frequency range, h addition, it prints space, absorption, sporadic E. 
and total losses for each test frequency within the available frequency range. 
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CRPL Propagation • Prediction Program 
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Radio system parameters are combined with geophysical and ionospheric 
characteristics to predict the performance of high-frequency sky-wave communi- 
cation circuits. The program computes maximum usable frequencies, optimum 
traffic frequencies, lowest usef’-l frequencies, probable modes of propagation, 
angles of arrival, circuit reliability, system loss, available signal-to-noise 
ratios, and field strength. Numerical representation is used for all parameters 
not expressed in closed mathematical form, such as world maps of critical fre- 
quency and atmospheric noise. The solution of the problem is divided into two 
parts - an estimation of the available signal, and an estimation of the required 
signal. The program was initially written for 7090-class computers and was 
translated at NEL to Fortran 63 for the CDC 1604- At a later date, NEL added 
CCIR report "'22 noise data to the program. In the 322 version the calculations of 
circuit reliability cannot be made. 


Collins Radio HF Program 

The program is similar to that used by NBS and yields comparable data. 

The differences are in the calculation of Lowest Usable Frequency (LUF) and 
auroral absorption. At present, the median noise levels are used to calculate the 
frequency that satisfies the loss equation; this frequency is called the LUF. Irv 
the NBS program for LUF, the loss equation is solved by trial and error for a 
reliability of '4) percent, taking into account changer of mode with a two-dimen- 
sional antenna gain function. In the NBS program, the additional loss is deter- 
mined by the F2-layer control point location. In the Collins method, the average 
of two absorption indices is calculated from geomagnetic coordinates of the rays. 

Input insertion of links is <inique. As many as ISO stations are permitted 
with as many as 200 combinations oi stations as links. 


Canada DRTE HF Program 

This program computes the maximum usable frequency (MUF) and lowest 
usable frequency due to E layer cm off of the F layer for a given mode or 
modes. The basic ionospheric data used for the prediction of the F2 and £ 
layer MUF are obtaineJ from DRTF’s manual prediction system. 


VLF Program (Papperf) - NELIAC MOD 7 {1604) 

The program solve*. the carth-ionuspher.- wave-guide mode equaiion In 
general, Budden’s formalism is employed, Soluluais (o Siokrs' equation in 
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{eras of third-order modified Hankel functions are employed to determine the 
reflection matrix solution to the problem of the three-layer boundary value 
Goasard’s solution for the upgoing wave is incorporated in the Papperf solution 
Required program inputs are the initial admittance matrix (Gosaard-Smith 
program), profile increment, integration limits, peth data, geophysical da fa, 
and control pi ametera. 

The program output lists the characteristic mode angles, phase velocity, 
attenuation, excitation factor, and modulus of the polarization vector- 


VLF Program (Smith/fiossard) 

A full-wave solution is made for electromagnetic propagation in a con- 
tinuous ionosphere with arbitrary parameters (electron density and collision 
frequency). Budden's solution is the basis. 

Inputs are initial conditions a high altitude, heigh: complex angle ot 
incidence, propagation angle, dip angle and magnetic field strength, freq-iency, 
collision frequency, and vertical profile of electron density. 

The basic program output is the reflection coefficient matrix expressed in 
polar coordinates as a function of height. 

The program employs a procedure for “nding an appropriate initial value 
for the reflection coefficient prior to performing the real ionosphere integration. 


DECO Program 

This program predicts the mean intensity of atmospheric noise for any 
frequency for which a wave-guide model of earth-ionosphere is acceptable. The 
wave-guide mode equation is used. The program assumes diet the mean noise 
intensity at a receiver mat be simulated by properly combining fields produced 
by a number of transmitters that replace the actual thunderstorm sources 

The program output is the summed field intensities for given locations and 
lanes. With an appropriate plotter worldwide contour maps of noise in'tnsities 
can be produced. 


DECO Program (NEL Variation) 

The basic DECO program will be modmed :o provide predictions for the 
phase arid amplitude of given transmutes for any location on the earth 
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Xift-Ftsks Ray Traci fStacforl/NEL Virsion) 

This prop am employs a simplified approximate procedure for tracing rays 
between a terminal pair, it provides an assessment of possible bop structures, 
using a zeroing-in procedure to reject all rays not within tlOO km of the receiving 
point. The ionospheric model is keyed to the CRPL predictions, tut represents 
the layers as psrabolir and concentric. The magnetic field effects are not 
included. 

The output includes the path modes, frequency, take-off angles, great- 
circle path length, travel time atoug me ray path, ionospheric absoiption loss 
(only) along lOute, and the maximum usable frequency. 

The inputs required are the terminal coordinates, the appropriate CRPL 
predicted ionosphere or a measured set of ver'icai soundings, declination of sun. 
frequencies, range take-off angle aperture, and time. £, layers may be included 
“after the fact.” 

Separate auxiliary programs provide for tape-loading the CRPL ionosphere, 
and for plotting die outputs on the 160 - A printer. 


Day Trace Program '.Shoddy) 

The Haselgrove equations are used 10 trace ray paths in three dimensions 
in a model ionosphere. The ionospheric model is a three-dimensional combina- 
tion of parabolic layers and CRFL (Callet-Joncs) world maps. It traces both 
ordinary and extraordinary rays. No special provision has been made ro use 
values of frequency below the gyro frequency. 

Input parameters are ictminal coordinates or azimuth and maximum dis- 
tance, eleva’ian angle, frequency, yc?r, month, and hour. 

Outputs include great-cirde oath length, ray angle, ray uughr, and 
geographic coordinates at each computed mini. 

The pro&am permits ready substitution of alternate ionospheric models. 
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APPENDIX C: DATA TYPE 


Data Type 

Average Prices 

of Navy Electronic Equipment 


General Characteristics 

of AN type Electronic Equipment 

by AN designation 


Equipment Characteristics 
and some price information 


AND SOURCE 


AMCP 706-131 


Para Source 

Bureau of Ships NavShips 92, 363(B), 

Index to Bureau of Ships Controlled Elec- 
tronics Equipment (F Cognizance) 15 
March 1963 

Bureau of Ships NavShips 900, 125(D), 
Nomenclature Assigned ;o Naval Elec- 
tronic Equipment , CONFIDENTIAL, 

August 1%3 

Bureau of Ships NavShips 94,200.01, 
Directory of Communication Equipment , 
SECRET, April 1964 

Bureau of Ships NavShips 94,200.0100, 
Directory of Classified Electronics Major 
Units , CONFIDENTIAL, January 1964 

Bureau of Ships NavShips 94,200.1, 

Section 1, Directory of Communication 
Equipment , n.d. 

Bureau of Ships NavShips 94,200.1, Section 
2, Directory of Communication Equipment , 
n.d. 

Bureau of Ships NavShips 94,200 i i 
tion 3, Directory of Communication Equip- 
ment , n.d. 

Bureau of Ships NavShips 94,200. 1, Sec- 
tion 4, Directory of Communication Equip- 
ment , n.d. 

Bureau of Ships NavShips 94,200.1, Sec- 
tion 5, Directory of Communication Equip- 
ment, n.d. 


Bureau of Ships NavShips 94,200.1, 
Section 6, Directory of Communication 
Equipment , n.d. 

Bureiu o! Strips NavShips 94,200.1, 
Section 7. D irectory of Communication 

Eiuip.p-*i . 4 , . .t'. 
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Equipment Characteristics 
and specification VTBf value 

Equipment Cost, 

Quantity and Date of procurement 


Electron c Equipment 
MTBF and MTTR values 


bureau of Ships NavShips 94,200-1 
Section 8, Directory of C ommunitatirw 
Equipment , n,d. ' 

Bureau ol Ships NavShips 94,200-1, 
Section 9, Directory of Communication 
Eq uipment , n.d. 

Ships Specifications 


F Cognizance 
Material Control Branch 
(Code 6627) 

Fleet Electronic Effectiveness Branch 
(Code 6678) 
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APPENDIX D: MILTIPLE-RESIESSIIN ANALYSIS * 


Regression analysis presupposes that some relationship exists between a 
dependent variable Y and one or more independent variables Xj, X; — X„. 
The simplest case is approximated by the linear equation of the font 

n 

0, 

i- 1 

f} t are the parameters i a one-dimensional space generated by the regression 
plane. The quantity e' represents the random or simple error in the variation of 

Y not accounted for in the regression plane. Genual transformations such as 
X{ - 1/X,- « X 2 ’ ~ X 2 2 will yield an equivalent form of the foregoing equation. 

The true values of constants #q, 02. — /3„ can never be determined. 

However, estimates of these constants can be obtained from m observations of 

Y and corresponding X,- values. A simple way of writing the m observations is 
in the form of a table as shown below: 



X’i 

x 2 " * * 

■ - - - X„ 

Yi 

*11 

*12" --- 

a ln 

y 2 

*21 

21 2 2 * * " “ 

--- a 2 „ 

Ym 


*m2 

- * ■ 


The linear estimating equation then has the form 

PI 

v = vl 

i — 1 


where 

a. is the regression coefficient and is an estimate of the true but unknown 
coefficient /3,- 

e is the residual of the true Y about the regression plane 
X,- is the independent variable 

Several assumptions are made about the independent and dependent variables 
that permit significance tests and confidence interval estimates to be made. 

These assumptions also lend themselves to the least-squares method of esti- 
mating the value of a/. 

These assumpti ons ate 

1. AllX.-'s are fixed variables (that is, there are no probability distributions). 
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2. Ill" Y's ire all nor wall* and indpendently distributed about the mean 

n 

(a Q f ^ a. X, )*ith variance <» 2 , 

i = l 

The least-squares method minimizes the sum of squares of deviation (G) from the 
estimated regression plane. 


m 



Graphical procedures are used to determine the form an< ! transformations re- 
quired in the linear regression equation. 

Multiple-regression analysis makes use of several statistical tests to 
determine the significance of coefficients and equations, The F-ttst determines 
whether the fmiu of tire equation i» statistically significant. The t-test checks 
the significance of the partial-regression coefficients The multiple-correlation 
coefficient R gives the degree of correlation between the dependent variable 
Y and the independent variables Xi* X 2 . X 3 — X n . A more detailed discussion 
o f 'hese tests follows. 


Multiple-Correlation Coefficient R 

The square of the multiple-correlation coefficient is defined as the frac- 
tion of the total variance of Y which is contributed by its regression upon the 
variables Xj. X 2 . "*X„. 

n m 

I Iv, 

,= i j - i 


" ) 

i = i 

The foregoing is obtained by expanding equation Dl and grouping into total sum 
of ?nd »*><* sum of squares to regression {nume r atoi Y 

A value of zero gives no correlation between Y and Xj. X 2 , — X n , where- 
as a value of l means all sample points lie on the regression plane. 



F Ratio Tesi 

The F-test determines whether die form of the regression equation is 
suti* really significant by comparing a calculated F value with a critical f for 
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(n) degrees ol freedom at a preselected significance (eve! c-f <dpfca 

o 2 I \ 

F = - — 5 - 

(i'R 2 ) (n) 

If the calculated value of F is greater than the critical value of F, then the null 
hypothesis that all a- t = 0 is rejected and the overall regression is judged to be 
significant for the alpha significance Wei. 

The F'test compares the sum of the squares due to regression with the sum 
cl squares due to error. 


Student f Distribution 


The variable t has the Student t distribution. 




where 

X - is the arithmetic mean of the data selected for a random sample 

of sire n r 

» = is the standard deviation of this random sample, and s/Jn is the 

standard error, s’ of X 

^ = is the arithmetic mean of all the values composing a normal popula- 
tion that has a su.tdard deviation j. 

If the calculated value of t exceeds the critical value of t a for the signi- 
ficanre level selected and m-n-' degrees of freedom, it can be said there is 
probability a that the actual divergence of the sample mean occu^cd simply by 
chance. 

For n greater than 30, the normal distribution gives a sufficiently precise 
approximation; for n£ 30 the t distribution should be used. 

The level of significance indicates the probability of obtaining a value of 
t outside the range of ±t (critical), for the degrees of freedom from 1 to 30, purely 
as a result of random sampling variation. 
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APPENDIX l: METHOD OF STEEPEST DESCENT 


the method of steepest descert is an optimization procedure that wit! 
locate an extreme value (in this case a minimum) of a criterion function. The 
method can be extended to functions with constraints. Changing the sign of the 
criterion function inteichanges the roles of the extrer *. values. For example, 
the minimum of F (x,y,x) is the maximum of - F(x,y,z). 

The method of steepest descent uses successive approximations to find an 
extreme value of the criterion function. Each new point P l + , is determined 
f , the expression 


. - A / if if *A\ 

\<?x ’ <Jy dz t 


l »+ 1 ~ ^ X i+ l.^i+ t, '! + l . , 

. , / ' 7 Kvi) 

3F 3 F dF\ 

In this expression ( — — — I is the gradient of F (denoted V F ) and is a 

\dx 3 y dz J 

vector in the direction of greatest increase oi F 

That the gradient is in die direction of greatest increase can be seen 
from the following argument: 

3F 3F 3 F 

uF = — d* + — d\ + — dz 

3x 3y 3: 

= VF-dP 

= |VF||dP|cos 6 

This expression for the differential or increment in F is greatest when 0 is 0: 
that is, when the gradient and the increment in the parameter vector are co- 
directional. 

The constant A (A>0) ts a scalar indicating the step size for the next set 
of coordinates in the direction of the gradient. The use of ~A Vf indicates that 
the path P 5 P 2< . . .. P„ leads ir* the steepest direction to a maximum of -F- 
th.it i< , a r The*-. -• "method of steepest descent.’* 

Following the selection of a new point, the criterion function ts evaluated. 
H F ( x i+ i, >'»+ t, : i+ i) < F (x, \j Zj) and the variables remain vithin present 

bounds, the point P ( + s becomes the new point of departure for finding 
If not. a new a is chosen. Tuere are several ways of choosing A, giving rise 
to variations of the method 

When the magnitude of the gradient becomes zero, it is concluded that F 
has reached a minimum. However, three things can happen to contravei c * valid 
solution: 

|. the minimum may be a local minimum, 

2 Ihe poini may he on a ledge, or 
). the point may be a saddle point. 

In either oi cases 2 or T, further variation of the variables would produce 
a further decrease in F. 
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The method of steepest descent is adaptable to problems witf constraints 
of two types: 


{(x.y.i)-a (i) 

(»' 

A constraint of type (i) can be written fix, y, z) - c = 0 so that z is defined as an 
implicit function of x and y, 

* = 4 > 

Then F(x, y, #x,yj) - Gfx.y), and 


where 



dG _ if + if ££ and — - — + — i z - 

dx dx dz dx dy 3, dz dy 


The partial derivatives— and — are evaluated with the aid of a theorem or 
dx dy 

the differentiation of rn implicit function that states that 


il 

dz dx 



if 


t 0. and similarly for il 
dz dy 


Each constraint equation has the effect of eliminating one variable from the 
problem, in the case discussed, the variable eliminated is z, since the function 
f specifics it as dependent upon x and y. This is the key to the problem with 
constraints of type (ii). As long as the constraint is satisfied, the problem is 
treated as one with no c. u ;oon as the constraint is vrolaicd by some 

taint (if, y t , z t ), the equality sign »; assumed to hold, and the problem is treated 
as one with a type (i) constraint. 

The method of steepest descent has one major liability; namely, the local 
minimum nearest the initial point (x t y t , a j > will be found. If die function has 
more than one best minimum, the true (global) minimum may be missed. The 
programmer should have some idea of what answer to expect in order to eliminate 
possible spurious answers by a wise choice of the initial value; o( i , y, and z ■ 
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APPENDIX F: LIST OF SYMBOLS 

in most cases in this report the 
relationship between capital and lower- 
case symbols is: F = 10 'og 10 f 


B - Ba^jwidth relative to 1 kc/s 
c - Number of channels 

Cf ■* Confidence factor 

D = Frequency shift - ,„.5 c/s 

E„ = Equivalent vertical'y polar- 

ized ground-wave root-mean- 
square noise field strength 
in dB above l^V/m for 1-kc/s 
bandwidth 

F - Effective receiver noise 

figure 

F u = Effective antenna noise 
figure 

F c = Effective receiver antenna 
coupler noise figure 
f 0 “ Cento frequency of frequency 

«nge h', 

Jm - Band width of tow-pass filter 

f r ~ Receiver noise figure 

ft - Transmission line noise figure 

f x - Operating frequency in x units 

of c/s 

fbw = Frequency range 

G r = Receive antenna gain 

G, = Transmit antenna gain 

hf = High frequency 

K s Boltzmann constant 1.38 

x 10-23 g/*K c/s 
L p -■ Propagation path loss 

m = Order of frequency diversity 

N 0 = Noise power per' 1-c/s band- 

width 

P, = Bit error rate 

P, = Average transmitter power 

Q = Quantity of units 

R = ST 0 /N 0 = normalized post 1 

detection S/N ratio 
r c( ~ Transmit am-nna coupler 
VSHR 

RPRE= Preelection S/N ratio 
s Receiver stal ility, PPM 
2 t = Transmitter stability, PPM 

T = Temperature fK) 

T 0 = Baud length (pulse length) 

vlf = Very low frequency 

cr ~ Standard error 
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Alternative Systems 

Identify Basic Alternatives, 2-12 to 2-15 
Synthesize Alternative Systems, 2-10 to 2-17 

Alternatives, 1-5 

Application of SA/CE, 1-12 to 1-16 

Analytic Models, 3-2, 3-28 to 3-30 

Army force Development Plan (AFDP) , 2-1 

Assumptions, 2-6, 2-26, 2-69 

Availability, 2-21 

Calculation of, 2-J3, *J-40, £-4l 

Calculus of Variations, 3*30 

Capability, 2-23 

Calculation of, 2-3*1, 2-39 

Characteristics 

Develop Hardware Characteristics, 2-17 to 2-l8 
Combat Development Objective Guide (CDOG), 2-3 
Concept Formulation 

Application of SA /CE during, 1-14 to 1-16 

• 

Contract Definition 

Application of dA/CS during, 1-14 to 1-16 
Prerequisites to, 1-14 to 1-15 
Requirements for, 1-15 

Cost Analysis, 1-9, 2-43 to 2-54 

Cost Analysis Problem, 2-51 to 2-54 
Cost Categorization, 2-44 
Cost Commeneurability, 2-49 

Cost Estimating Relationship, 2-48, 3- 31 to 3-3;, 

Cost of Maintenance Manpower, 2-53 

Cost of Supply Manpower, 2-54 

Cost Model, 2-25 

Cost Uncertainty, 2-49 

Experience Curves, 3-39 

Incremental Costing, 2-43, 2-47 

Investment Costs, 1- 44, 2-45, 2-46 

Material Cost3 at Supply, 2-53 

Operating Coats, 2-44. 2-46 

RAD Costs, 2-44, 2-45 

Sensitivity Analysis, 2-44, 3-41 to 3~**5 

The RAND Method, 2-43 
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Cost-affectivenese 

Array of Cl Cbareeterietlea- 2-71 
Application of, 1-12 to 1-16 

Application of, to Concept formulation and Contract 
Definition, 1-14 to 1-16 
Background and Elatory of, 1-5 to 1-7 
Combination with Systems Analysis, 1-7 
Definition of, 1-2 
ntaaplea of Ct Curves, 2-72 
limitations of, 1-17 to 1-19 
Methodology of (Detailed), Chapter 2 
Methodology of (General) r . 1-7 to 1-12 
Requirement for During Life Cycle, 1-12 to 1-13 
See also, Cost 
See also, Zffectiveness 

Cost Sensitivity Analysis, 3-41 to 3-45 

Criteria, 1-9 

Data 

Collection of, 2-50 

Statistical Analysis of Vest Data, 4-28 to 4-?’ 

Decision Theory, 3-2 f 3-30» 3-3'- 

Definitions of Cost -Effectiveness, 1-2 

Definitions of Systems 'nclyiia, 1-1, 1-L 

Dependability, 2-22 

Calculation of, 2-33, 2-40 

DynaailJ Prograrmirg, 3 2, 2.-16 to ?.-l? 

Zconaelc Welfare Theorv 

As it Relates to coet-lffoctivensss, 1-6 

Effectiveness 

Availability, 2-21 

basis for Evaluating, 2-18 to 2-24 

Capability, 2-23 

Define Measures of, 2-2'» 

Dependability, 2-22 

Effectiveness aquations, 2-29 to 2-30 

Effectiveness Model, 2-25 

KLeuc.it « of Zf.'ectlv.: .ess, 2-wO 

System Effectiveness, 2-20 

System Effectiveness Problem, 2-31 to 2-42 

Experience Curves,. 3-39 to ^-41 

Oeae Theory, 3-2, 3™i7 to 3-22 
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History of Cost-Xffsctivensss, 1-5 to 1-7 
History of Systsms Analysis, 1-2 to 1-5 
Incremental Costing, 2*43, 2-47 
Infomstion Theory, 3.2, 3.22 to 3-28 
Inventory end Replacement, 3.2, 3-9 to 3-11 
Investment Cost, 2-44 


Kolmogorov -Smirnov Test, 4 - 4 o 


Lanchester's Equations, 3-28 to 3-30 
Leverage Effects, 2-66 to 2-67 


Limitations of SA/CS, I-17 to 1-19 
Linear Programming, 3-2, 3-12 to 3-16 


Maintainability 

Point and Internal Estimate* of, 4-30 


Methodology of SA/CE 
General, 1-7 to 1-12 
Detailed, Chapter 2 
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Mission Profiles, 2-7 to 2»$ 

Models, 1-9 

Analytic Models, 3 - 2 , 3-28 to 3-30 
Cost Model, 2-25 
Decision Model, 2-62 to 2-69 
Effectiveness Model, 2-25 
Exercise of, 2-55 te 2-62 
Formulate, 2-25 to 2-28 
Gaming Models, 2-28 
Mathematical, 2-27 
Operational Exercise, 2-28 
Simulation Models, 2-28 
Types of, 2-27 

Objectives, 1-8 

Operating Coats, 2-44 
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Optimization 

Criterion, 2-62 

List of Techniques, 2-66 

Output Results, 2-70 to 2-73 


Performance Parameters, 2-9 to 2-10 





Prerequisites to Contract Definition, 1-14 to 1-15 

Probability 

Basic Laws, 4-4 to 4-9 
Definition, 4-2 
Distributions, 4-9 to 4-2? 


Qualitative Material Development Objective (QMDO), 1-12, 2-3 


Qualitative Material Requirements (QKR), 1-12, 2-3, 2-5 


Queueing Theory, 3-2, 3-3 to 3-7 
Reliability 

Applications of, 4-25 to 4-28 

Confidence Limits for Reliability Functions, 4-36 
Point and Internal Estimates of, 4-30 


Requirements 

Defining, 2-5 to 2-7 


Research and Development Costs, 2-44, 2-45 


Sensitivity Analysis, 2-43, 2-62, 3-41 to 3-45 


Sequencing and Markov Processes, 3-2, 3-7 to 3-9 

Simulation, 3-2, 3-3 
Models, 2-28 

Small Development Requirements (SDR) , 2-3 
State-of-the-Art Analysis, 2-16 


System 

Total System, 2-10 tr> 7-11 
See also. Alternative Systems 


Systems Analysis 

Application of, 1-12 to 1-lu 

Application of, to Concept Formulation and Contract Definition, 
1-14 to 1-16 

Background and History of, 1-2 to 1-5 
Combination with Cost Effectiveness, 1-7 
Definition of, 1-1, 1-4 
Limitations of 1-17 to 1-19 
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Byatma Analyst* (continued) 

of (Detailed), Chapter 2 

K«n»n?+ 0gy 0t ( 0en ® r * 1 )» 1-7 to 1-12 
Nonmilitary use of, 1-5 

Requirement for during Life Cycle, 1-12 to 1-13 
Trade-Off Analyses, 2-57 to 2-62 

Interdependence Between Trade-Off studies and Prolect Obiertiv* o at 
P erformance Parameter .'rade-Offs, 2-50 J Objectives, 2-61 

Performance vs. Cost Trade-Offs, 2-55 

Uncertainty, 2-26, 2-69 

Confidence Intervals, 3-32 to 3-35 
Cost Uhcertalnty, 2-49 to 2-50 
Expressions of, 2-57 
Risk and Uncertainty, 2-64, 2-65 

Weibull Distribution, 4-Co 


WSEIAC, 2-19 

Availability, 2-21 

Capability, 2-23 

Dependability, 2-22 

Effectiveness Equations, 2-29 to 2-30 
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